-- Logs begin at Wed 2022-10-19 12:47:14 UTC, end at Wed 2022-10-19 14:06:19 UTC. -- Oct 19 13:15:49.420087 dv-cinder-861883-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 19 13:15:49.427506 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 19 13:15:49 2022] *** Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: nodename: dv-cinder-861883-1-iscsi Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: machine: x86_64 Oct 19 13:15:49.429552 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: clock source: unix Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: pcre jit disabled Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: detected number of CPU cores: 4 Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: current working directory: / Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: detected binary path: /usr/bin/uwsgi-core Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: your processes number limit is 47680 Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: your memory page size is 4096 bytes Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: detected max file descriptor number: 2048 Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: lock engine: pthread robust mutexes Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: thunder lock: enabled Oct 19 13:15:49.429956 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 19 13:15:49.430588 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 19 13:15:49.471482 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: Python main interpreter initialized at 0x556628d909c0 Oct 19 13:15:49.471482 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: python threads support enabled Oct 19 13:15:49.471482 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: your server socket listen backlog is limited to 100 connections Oct 19 13:15:49.471482 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: your mercy for graceful operations on workers is 90 seconds Oct 19 13:15:49.471855 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: mapped 403077 bytes (393 KB) for 2 cores Oct 19 13:15:49.471939 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: *** Operational MODE: preforking *** Oct 19 13:15:49.472216 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: *** uWSGI is running in multiple interpreter mode *** Oct 19 13:15:49.472216 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: spawned uWSGI master process (pid: 81524) Oct 19 13:15:49.472427 dv-cinder-861883-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 19 13:15:49.473813 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: spawned uWSGI worker 1 (pid: 81537, cores: 1) Oct 19 13:15:49.473813 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: spawned uWSGI worker 2 (pid: 81538, cores: 1) Oct 19 13:15:49.473813 dv-cinder-861883-1-iscsi devstack@keystone.service[81524]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 19 13:15:53.791300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 13:15:53.791300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: from cryptography.utils import int_from_bytes Oct 19 13:15:53.792639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 13:15:53.792639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: from cryptography.utils import int_from_bytes Oct 19 13:15:53.793681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 13:15:53.793681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Oct 19 13:15:53.794133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 13:15:53.794133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Oct 19 13:15:53.844528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 13:15:53.844686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 13:15:53.844779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 13:15:53.844859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 13:15:53.844921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] command line args: [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 13:15:53.844984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] command line args: [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 13:15:53.845153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 13:15:53.845248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 13:15:53.845332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ================================================================================ {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 13:15:53.845410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ================================================================================ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 13:15:53.845571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] admin_token = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.845656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.845817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] config_dir = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.845902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_dir = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] config_source = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_source = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] debug = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] debug = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.846985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.847926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_config_append = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_config_append = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_dir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_dir = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_file = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.848984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.849925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.850954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.851949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] publish_errors = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] publish_errors = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.852980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] standard_threads = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.853997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] standard_threads = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_journal = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.856982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_json = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] use_journal = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_stderr = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_syslog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] use_json = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.857563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.857563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.857563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.857986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] use_stderr = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.858104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.858217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] use_syslog = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.858329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.858442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 13:15:53.858663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.858782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.858895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.859961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.860947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.861983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.862975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.863939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.864923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.865984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.866969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.867942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.868951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.869973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.870921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.871951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.872967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.873919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.874055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.874161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.874258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.874359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.874455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.877929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.878995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.879881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.880926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.881940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.external = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.882997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.external = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.password = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.token = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.883960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.password = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] auth.token = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.884958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.885975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.886890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.887842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.888912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889261 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889819 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.889960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.890938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.891986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.892931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.893997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.894977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.895909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.896973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.897899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.898898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.899960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.900910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.901530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.901654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.901777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.901905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.902859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.906988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.907941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.908967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.909948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.910966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.911977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.912921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.913956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.914878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] role.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] role.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.915973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.916993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.917972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.918992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.919941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.920924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.921906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.922879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.923941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.924901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.925851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.926978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.927933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.connection = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.928933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.929938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.930909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.931960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.932821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 13:15:53.932902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.933084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 13:15:53.933254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 13:15:53.963051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.963197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.963277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.963435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.964024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.964106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.972317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:53.972413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.005008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.005237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.005485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.005720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.067833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.069410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.117644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.117873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.118415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.118775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.272888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.273958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.274041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.274173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.274394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.300980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.301612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.312965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.313318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.327978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.328729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 13:15:54.345296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.345501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 13:15:54.345587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.345774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 13:15:54.345855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 13:15:54.346121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 13:15:54.346594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.346952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.347107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.347192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.347460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.356670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.356949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.357925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.364450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.364852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.367361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.367647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.367853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.367938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.368983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.369065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.369250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.369331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.369519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.370879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.371152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.393496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.394255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.394983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.395269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.395759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.396046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.399886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.400176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.400456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.400764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.401078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.401386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.406776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.407066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.407799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.408146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.412621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.412921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.413206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.413488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.413720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.413804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.414881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.415161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.415483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.415712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.429797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.430077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.431174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.431460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.435152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.435800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.438305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.438632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.438998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.439407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.442757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.443508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.445796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.446573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.449580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.449881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.450179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.450382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.450470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.450716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.450803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.451006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.451097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.451293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.451583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.451862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.466823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.467112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.467380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.467643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.467948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.468140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.468413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.468682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.474420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.475522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.475980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.477054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.477500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.477777 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.478597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.478878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.480608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.480894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.481176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.481460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.481580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.481857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.481941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.482137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.482420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.482881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.492922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.493208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.493489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.493772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.494068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.494160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.494354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.494444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.495173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.495259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.495460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.495841 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.496138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.496421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.511033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.512407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.513934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.515350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.516818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.518216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.518300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.519738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.519882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.520015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.520412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.521365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.521653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.522061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.526800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.528245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.528435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.529890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.531148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.531546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.531832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.532108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.532910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.533311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.533604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.533886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.539871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.540170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.540454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.541728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.542023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.542307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.547308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.547594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.547876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.548347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.548445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.548677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.548948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.549226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.549316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 13:15:54.549502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.549586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.549768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.549860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.550960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.551170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.551444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.551722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.552008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.552280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.552548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.552810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 13:15:54.585895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556628d909c0 pid: 81538 (default app) Oct 19 13:15:54.588161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556628d909c0 pid: 81537 (default app) Oct 19 13:15:54.589188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467bafc-359e-4396-bc3b-1e52a5f0b52f None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:15:54.589475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467bafc-359e-4396-bc3b-1e52a5f0b52f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:15:54.589750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467bafc-359e-4396-bc3b-1e52a5f0b52f None None] PATH_INFO: `/v3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:15:54.593904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Wed Oct 19 13:15:54 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:02.164200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:02.164733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:02.164733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:02.169068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:02.174258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:02.180125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:02.180125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:02.245862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: INFO dbcounter [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] Registered counter for database keystone Oct 19 13:16:02.265155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG oslo_db.sqlalchemy.engines [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81538) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 13:16:02.271453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writer thread running {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 13:16:02.448838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG passlib.handlers.bcrypt [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81538) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 19 13:16:02.449220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG passlib.handlers.bcrypt [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81538) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 19 13:16:02.506844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:02.508421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:02.633901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 13:16:03.100229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:03.169942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: INFO dbcounter [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] Registered counter for database keystone Oct 19 13:16:03.191495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG oslo_db.sqlalchemy.engines [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81537) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 13:16:03.197711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writer thread running {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 13:16:03.314712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3c690-0cd8-4f4a-ac37-3263e84375c3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:03.319009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:03.319632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:03.319922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:03.320194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:03.328340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:03.379618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 19 13:16:03.379954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46a4df-0147-4e48-8a8d-a1c28802ba62 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:03.389262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Wed Oct 19 13:16:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 13:16:03.395183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:03.412237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2db0-d427-4946-9e0b-d32807fa189d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:03.416720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:03.418449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:03.418768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:03.419047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] PATH_INFO: `/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:03.420659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=cf3ff3a529704cf9a1e0bae3d4c303ad)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:03.472733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING keystone.common.rbac_enforcer.enforcer [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 19 13:16:03.473089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a873e0-13f0-462d-b27a-97a1ca2f95f2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:03.477587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:16:03 2022] GET /identity/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:05.582725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d306a-6e7b-4648-a0d4-93558d6529c1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:05.583246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d306a-6e7b-4648-a0d4-93558d6529c1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:05.583353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d306a-6e7b-4648-a0d4-93558d6529c1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:05.587631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:05.593534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:05.593956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:05.594098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:05.614644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:05.616177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:05.623979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 19 13:16:05.891604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d306a-6e7b-4648-a0d4-93558d6529c1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:05.903757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d306a-6e7b-4648-a0d4-93558d6529c1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:05.907865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:05.908457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:05.908738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:05.909095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:05.916712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:05.918545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a384dd-0968-464b-843f-70da083a7270 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:05.927220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Wed Oct 19 13:16:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 13:16:05.932457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:05.941421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d299b07-f1d7-4879-994c-61cdc84b0527 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:05.945778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:05.946400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:05.946720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:05.946996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] PATH_INFO: `/v3/users/94146067898240bfb358cef661f9b970` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:05.948304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] RBAC: Authorizing `identity:get_user(user_id=94146067898240bfb358cef661f9b970)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:05.950685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd325-b9a8-4990-9529-c3bfc0baeb8c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:05.952637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Wed Oct 19 13:16:05 2022] GET /identity/v3/users/94146067898240bfb358cef661f9b970 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:08.124666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ab18-ef66-47b5-a924-95cbe7aba4ee None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.125204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ab18-ef66-47b5-a924-95cbe7aba4ee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.125204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ab18-ef66-47b5-a924-95cbe7aba4ee None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.126641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:08.131686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.132078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.132237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.149978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:08.151186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:08.175613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b5a4-f380-40df-a041-93ae1c4b08f2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.175900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b5a4-f380-40df-a041-93ae1c4b08f2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.176181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b5a4-f380-40df-a041-93ae1c4b08f2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.177500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:08.182617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.182910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.183805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.225529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81537) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 19 13:16:08.225910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81537) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 19 13:16:08.232716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:08.276337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:08.277875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:08.377319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:08.499548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.523140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:08.602195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b86c6-2344-4f42-8192-fe772a59ec38 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:08.634715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:08.635342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.635623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.635900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.640848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.648317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:08.657274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:08.658241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:08.659569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:08.664008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.743475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84821a7a-07b4-49fd-8aed-cdbc04d191b8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:08.744021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:08.745822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:08.762034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674980d-53e2-4a0d-9be7-e11002f2d7ea None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:08.781224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:08.791589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.791589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.791589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.791589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:08.791589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:08.796294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:08.796927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.797208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.797484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.798151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:08.798452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:08.799843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:08.811865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:08.817327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.826720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:16:08.826720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:16:08.826720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [parameters: {'id': '946e0f08779d4d8dbcf55f32a037a644', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 13:16:08.826720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:16:08.845811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 13:16:08.846654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 13:16:08.847565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 13:16:08.848457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 13:16:08.849368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:16:08.850173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '946e0f08779d4d8dbcf55f32a037a644', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 13:16:08.851019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:08.851748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 8/16] 10.222.0.39 () {60 vars in 1137 bytes} [Wed Oct 19 13:16:08 2022] POST /identity/v3/domains => generated 190 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:08.852655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.915573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133303d5-75ad-4b0d-9275-adc4b803bb43 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:08.948619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:08.949774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.950056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.950339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] PATH_INFO: `/v3/users/94146067898240bfb358cef661f9b970` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:08.951509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] RBAC: Authorizing `identity:get_user(user_id=94146067898240bfb358cef661f9b970)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:08.952928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0cf0a-5f2f-406d-a1aa-9bd6c7398001 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:08.953023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:08.954545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:08 2022] GET /identity/v3/users/94146067898240bfb358cef661f9b970 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:08.960131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:08.975222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:08.987073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:08.987926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:08.988213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:08.988490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:09.006062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:09.007594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:16:09.023922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:09.024807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:16:09.025697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:16:09.025697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:16:09.025697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:09.025697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:09.031277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:09.052460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:09.067543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165200-7a4e-4302-aef7-01e0432d6a8b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:09.103393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:09.103958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:09.104243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:09.104519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:09.105663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:09.107172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:09.108882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:09.114181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:09.132663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7e0e2-7eb5-47c3-a7a6-0f84cbad9f56 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:09.166112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:09.166741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:09.167025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:09.167300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:09.167942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:09.168243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:09.169610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b85345-1af6-4e40-8951-ac84c80bfe82 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:09.187087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:09.193418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:09.214049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7282a-4d96-4bfd-a7bb-62edaaa323cb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:09.251396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:09.252083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:09.252363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:09.252640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] PATH_INFO: `/v3/domains/default/users/94146067898240bfb358cef661f9b970/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:09.254618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=94146067898240bfb358cef661f9b970, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:09.256093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc719e2-783e-4727-9eca-9fae94427634 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:09.463534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 11/21] 10.222.0.39 () {58 vars in 1361 bytes} [Wed Oct 19 13:16:09 2022] PUT /identity/v3/domains/default/users/94146067898240bfb358cef661f9b970/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:16:11.569985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d321-f2e8-4037-a97d-2ea497d15e33 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:11.570540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d321-f2e8-4037-a97d-2ea497d15e33 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:11.570540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d321-f2e8-4037-a97d-2ea497d15e33 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:11.571897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:11.578274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579feb0-4eed-4041-9924-0abf0b4a831f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:11.578735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579feb0-4eed-4041-9924-0abf0b4a831f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:11.578907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579feb0-4eed-4041-9924-0abf0b4a831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:11.603562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6579feb0-4eed-4041-9924-0abf0b4a831f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:11.611617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6579feb0-4eed-4041-9924-0abf0b4a831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:11.693706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:12.169474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39297-aae3-42cf-878b-b12e85caa226 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.169872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39297-aae3-42cf-878b-b12e85caa226 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.170054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39297-aae3-42cf-878b-b12e85caa226 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.171564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:12.176909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.184552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.184552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.189946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28080c-5391-427d-b56a-5967f323a346 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.190231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28080c-5391-427d-b56a-5967f323a346 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.190526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28080c-5391-427d-b56a-5967f323a346 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.191959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 13/25] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:12.206980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204a76-7599-4dc5-aee5-2fcb3aed025a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.207268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204a76-7599-4dc5-aee5-2fcb3aed025a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.207535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204a76-7599-4dc5-aee5-2fcb3aed025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.213231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:12.214448 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:12.234760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-32204a76-7599-4dc5-aee5-2fcb3aed025a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:12.236091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-32204a76-7599-4dc5-aee5-2fcb3aed025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:12.338158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:12.346358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 13/26] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:12.347738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:12.352466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 14/27] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:12.374273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c150f5-2874-408e-a247-1c207f4f240a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.374752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c150f5-2874-408e-a247-1c207f4f240a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.375131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c150f5-2874-408e-a247-1c207f4f240a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.376862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 15/28] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:12.388890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.389290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.389686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.421783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:12.423155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:12.426829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:12.530775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 16/29] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:12.574712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad487a-3e74-4970-9702-4ccab547881c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:12.633599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:12.634195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.634413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.634712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.645855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:12.651614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:16:12.668567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:12.669435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:16:12.670234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:16:12.670234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:16:12.670234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:12.671824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 14/30] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:12.677039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:12.712694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:12.808911 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:12.871648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f0e9f-3c09-4b62-bf55-7468bb0e2e42 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:12.898775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af055bc-2871-4ba3-9618-20c9e8b25806 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:12.907378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:12.907996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.908274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.908545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.909199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:12.909500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:12.910913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:12.922883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:12.923133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 17/31] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:12.924237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:12.979227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:12.979881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:12.980172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:12.980457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:12.981136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:12.982563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:13.040363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c77b7d-00f6-4b82-990d-cc3f6e37e061 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:13.075891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:13.076527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.076807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.077075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.089547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:13.095611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:13.096648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:13.097524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 18/32] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:13.098548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:13.128764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 15/33] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Oct 19 13:16:12 2022] POST /identity/v3/roles => generated 219 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:13.130103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:13.222279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:13.308641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75114f-d2e0-4907-8de1-274cd0dd0b47 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:13.323212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4a70c-8c25-4957-8dab-a4d8df036a1e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:13.326881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:13.360826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:13.361459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.361969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.362263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.363025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:13.364883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:13.372934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:13.373556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.373840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.374118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.374811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:13.376314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518c74-f0db-423a-8052-4fbb02922978 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:13.461738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 16/34] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Oct 19 13:16:13 2022] POST /identity/v3/roles => generated 225 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:13.462613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:13.579104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7543962-7b47-4787-858b-3c4d162908a3 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:13.588514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 19/35] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:16:13 2022] POST /identity/v3/projects => generated 301 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:13.589992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c17d-c552-4fd9-bac8-dfa26ae64ed8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.590286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c17d-c552-4fd9-bac8-dfa26ae64ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.590645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c17d-c552-4fd9-bac8-dfa26ae64ed8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.592192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 20/36] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:13.593237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724197b9-0224-48b6-80c9-b5a64386c2f1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.593619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724197b9-0224-48b6-80c9-b5a64386c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.593903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724197b9-0224-48b6-80c9-b5a64386c2f1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.595258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 21/37] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:13.597833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.598272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.598666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.630942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:13.631338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:13.632123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.632515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.632898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.633722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:13.634275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:13.634384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:13.635816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c704d28-a87a-4e56-8098-7a9d0d876a71 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:13.649445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 17/38] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:13.650323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.650725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.651077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.672730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:13.673979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:13.707027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:13.737368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 22/39] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:13.737368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:13.763593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 18/40] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:13.799674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:13.862347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477ebbc-3179-45c3-a147-4eb2db0de3f7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:13.897938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:13.898605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:13.898890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:13.899166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] PATH_INFO: `/v3/users/94146067898240bfb358cef661f9b970` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:13.900537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] RBAC: Authorizing `identity:get_user(user_id=94146067898240bfb358cef661f9b970)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:13.902682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:13.904468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 23/41] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:13 2022] GET /identity/v3/users/94146067898240bfb358cef661f9b970 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:13.909761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:14.016951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b870d1a-7b45-4115-8e6d-95acf27a051d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:14.020334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:14.052453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:14.053098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:14.053377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:14.053647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:14.054765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:14.056207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:14.057771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 19/42] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:14.058576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:14.089810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:14.127940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143cbcd-4d05-4473-a869-0509d24cae64 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:14.138781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:14.158296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae9bf-8830-41c4-bef6-e5a07e45b5a4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:14.161728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:14.162335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:14.162652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:14.162928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:14.163594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:14.164996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:14.241535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 24/43] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Oct 19 13:16:14 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:16:14.242411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:14.303729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:14.304401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:14.304688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:14.304967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:14.306274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:14.307818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:14.309479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 20/44] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:14.315322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:14.361849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493232ec-a312-4d55-be32-c060d255cd90 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:14.404229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:14.404767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:14.405045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:14.405340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:14.406055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.domain.id=default, user.id=94146067898240bfb358cef661f9b970) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:14.406366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:14.407899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5534f-cab6-4837-82a5-567df3114d32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:14.417545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 25/45] 10.222.0.39 () {56 vars in 1322 bytes} [Wed Oct 19 13:16:14 2022] GET /identity/v3/role_assignments?user.id=94146067898240bfb358cef661f9b970&scope.domain.id=default&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 530 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:14.424521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:14.427563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec01e1-fd37-4059-9346-9a580e3020ec None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:14.471062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:14.471723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:14.472005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:14.472332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:14.473033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:14.474549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a49f0-b000-49a3-bb89-c8ffeb477dd6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:14.551123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 21/46] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:16:14 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:16:16.657579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419d11-3cf3-4b35-95c6-e6d17b3c51b2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:16.658127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419d11-3cf3-4b35-95c6-e6d17b3c51b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:16.658127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419d11-3cf3-4b35-95c6-e6d17b3c51b2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:16.659495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 26/47] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:16.664688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:16.665077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:16.665259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:16.684142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:16.685322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:16.749551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 22/48] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:17.014166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419d11-3cf3-4b35-95c6-e6d17b3c51b2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:17.048534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:17.112531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419d11-3cf3-4b35-95c6-e6d17b3c51b2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:17.145057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:17.145680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:17.145962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:17.146237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:17.147371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:17.148818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3567f-f205-4c75-b01d-03759a8b948d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:17.150350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 27/49] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:17.156053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:17.235091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:17.252555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af898e4e-fc7c-49a7-8aff-e9493bffb81e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:17.284852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:17.285391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:17.285680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:17.285957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:17.286644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:17.288123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d57b-5ddb-44e4-a0e9-84f2533503ae None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:17.369689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 23/50] 10.222.0.39 () {60 vars in 1140 bytes} [Wed Oct 19 13:16:17 2022] POST /identity/v3/projects => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:19.504310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0d15-9972-420f-84d4-948606669061 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:19.504822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0d15-9972-420f-84d4-948606669061 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:19.504907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0d15-9972-420f-84d4-948606669061 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:19.506196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 28/51] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:19.511827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:19.512204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:19.512398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:19.530782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:19.531953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:19.596440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 24/52] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:19.866715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0d15-9972-420f-84d4-948606669061 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:19.969326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0d15-9972-420f-84d4-948606669061 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:19.970835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:20.113391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:20.114071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:20.114360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:20.114673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:20.115778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:20.117268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577e0bf-ec80-4aab-8e29-f7ccb2de4bf4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:20.118817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 29/53] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:20.124997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:20.185578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:20.223585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07606-5900-4f26-8908-ae12195fa5b3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:20.256920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:20.257520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:20.257803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:20.258079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:20.258761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:20.260207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418817b8-1ec3-41db-887d-edcd9e35a4ad None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:20.365691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 25/54] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:20 2022] POST /identity/v3/users => generated 275 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:23.434646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1479-967e-4205-a363-9866ff30fe9a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.435667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1479-967e-4205-a363-9866ff30fe9a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.436149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1479-967e-4205-a363-9866ff30fe9a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.438009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 30/55] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:23.443384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b10df-6922-40db-9fa3-3c4a856040d2 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.444161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b10df-6922-40db-9fa3-3c4a856040d2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.444633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b10df-6922-40db-9fa3-3c4a856040d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.475499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cc1b10df-6922-40db-9fa3-3c4a856040d2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:23.477041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc1b10df-6922-40db-9fa3-3c4a856040d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:23.580861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 26/56] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:23.656172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190d8a-0a5c-455c-a8f1-9ab35b1981a2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.656785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190d8a-0a5c-455c-a8f1-9ab35b1981a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.657275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190d8a-0a5c-455c-a8f1-9ab35b1981a2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.658847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 31/57] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:23.674961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.675515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.675984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.703593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:23.705082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:23.728246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a31aa-7115-439c-a6b9-0bdde79c3a74 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.728796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a31aa-7115-439c-a6b9-0bdde79c3a74 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.729316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a31aa-7115-439c-a6b9-0bdde79c3a74 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.731865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 32/58] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:23.736941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e0c59-053f-4d7e-8bd6-05c67e2d67fe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:23.737484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e0c59-053f-4d7e-8bd6-05c67e2d67fe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:23.737955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e0c59-053f-4d7e-8bd6-05c67e2d67fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:23.769068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b75e0c59-053f-4d7e-8bd6-05c67e2d67fe None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:23.770269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b75e0c59-053f-4d7e-8bd6-05c67e2d67fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:23.786868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:23.819679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 27/59] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:23.884945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 33/60] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:24.022723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.159920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238db2c-752f-4483-9836-3ff13ca56b7e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.160303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238db2c-752f-4483-9836-3ff13ca56b7e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.160511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238db2c-752f-4483-9836-3ff13ca56b7e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.160811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5c449-a744-4731-88cc-fa58cb71aa71 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.161994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 34/61] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:24.166845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.167167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.167461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.185820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:24.187164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:24.220602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.221244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.221527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.221801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.234798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:24.239721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:24.240567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:24.241338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 28/62] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:24.241960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.288750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.290174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 35/63] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:24.291343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.322474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.377056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673e751-074d-434d-aa50-dbe230cd5544 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.396816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64668ba-235d-4c3a-bede-4f512e8560e4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.427137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.427795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.428077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.428350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.444111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.444756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.445002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.445274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.445946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:24.446247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:24.447706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:24.450784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:24.456831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:24.457693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:24.458487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:24.461647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 29/64] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:24.462736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.475701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 36/65] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:24.476564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.604401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.608808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0362027-4c00-4295-8dab-a88cedde07e5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.613867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.624836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8919-f83f-461c-9531-f3adc439bd77 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.643595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.644226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.644502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.644771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.657507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:24.663215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:24.664094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:24.664843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:24.666671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 30/66] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:24.667975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.669512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.670137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.670425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.670736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.671541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:24.672119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:24.673536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053da4-18f1-472f-a9ab-521934cfe23c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 37/67] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784e2e-a481-42cf-b890-498c7beee6e4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784e2e-a481-42cf-b890-498c7beee6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784e2e-a481-42cf-b890-498c7beee6e4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 38/68] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:24.686435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784e2e-a481-42cf-b890-498c7beee6e4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.784670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fd8a-40e6-4a0d-aadf-9df7a319a600 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.805368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784e2e-a481-42cf-b890-498c7beee6e4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.823455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.824090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.824377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.824657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.837999 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:24.839551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:24.839853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.840492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.840776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.841054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.841185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 31/69] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:24.841938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.851600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:24.857111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:24.857942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:24.858737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 39/70] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:24.860572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.860672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:24.876812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:24.940257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c5c7c-4c11-4e4d-bad2-e1d625b5034b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.958300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9543e-dd19-41df-a6db-5cb653be5a41 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:24.973277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.973885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.974169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.974443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.975134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:24.975437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:24.976815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56a584-da89-4974-998c-d8436946f873 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:24.985211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 32/71] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:24.986344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.986664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.986943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.992821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:24.993454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:24.993738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:24.994012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:24.995173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:24.997170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.003344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 40/72] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.003344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.007360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:25.008558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:25.073702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 33/73] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:25.074576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.087423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.088010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.100685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d2f9a-ff56-4e04-a3a3-2591fd929430 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.134068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a089-47f0-401c-a560-4990e456586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.134669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a089-47f0-401c-a560-4990e456586b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.134955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a089-47f0-401c-a560-4990e456586b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.135233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a089-47f0-401c-a560-4990e456586b None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.136400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a089-47f0-401c-a560-4990e456586b None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.140799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a089-47f0-401c-a560-4990e456586b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.140799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 41/74] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:24 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.140799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a089-47f0-401c-a560-4990e456586b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.192481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dc56-8798-483a-9fcc-80a7b25e9214 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.232593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.233215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.233500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.233782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.234459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:25.234812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.236216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.244929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 34/75] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.245571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.253214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a089-47f0-401c-a560-4990e456586b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.291831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.293130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.293414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.293695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] PATH_INFO: `/v3/users/94146067898240bfb358cef661f9b970` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.294866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] RBAC: Authorizing `identity:get_user(user_id=94146067898240bfb358cef661f9b970)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.296323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.297915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 42/76] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/users/94146067898240bfb358cef661f9b970 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.298687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.320713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.324674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.351111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a166f6f-9ec8-416a-965a-f042df4934e2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.384938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.385499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.385784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.386588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.387712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.389163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.390828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 35/77] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.391686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.406634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbb347-ced1-421e-bba3-1894d848a630 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.439620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.440278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.440559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.440835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/aeec84e937e04e0cbd930c7b84cb139d/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.442758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, user_id=aeec84e937e04e0cbd930c7b84cb139d, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.444213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.491658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c18cf5-c72e-4092-81b1-02e233c0bcd1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.512476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 43/78] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:25 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/aeec84e937e04e0cbd930c7b84cb139d/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:25.513200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.536483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.537140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.537427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.537704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.538968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.540535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.542271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 36/79] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.543106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.556602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.610347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.633756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e754-9a95-4279-86a1-5493950ccd34 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.659800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0ace3-06a6-4a36-a84e-5dfe4aacbfe6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.672931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.673705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.673986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.674266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.675454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.676907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.678772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 44/80] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.679327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.706726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.707485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.707774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.708048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.709274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.711131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.712981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 37/81] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.713702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.786387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667dcc-5981-4c87-9342-382290147c27 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.799363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.821050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.821716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.821997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.822271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/aeec84e937e04e0cbd930c7b84cb139d/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.824062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc3513-42ad-4451-affe-7aa0725f09da None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:25.825023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, user_id=aeec84e937e04e0cbd930c7b84cb139d, role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.826594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.859280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:25.859919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:25.860204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:25.860484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] PATH_INFO: `/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:25.861607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] RBAC: Authorizing `identity:get_project(project_id=289b8d43f2774ba08aeee54b5e9f0bf1)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:25.863112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:25.869608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 38/82] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:25 2022] GET /identity/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:25.869608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.920962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 45/83] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:25 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/aeec84e937e04e0cbd930c7b84cb139d/roles/2aaa59501e4146f5adce6e3872912797 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:25.922150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:25.939395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:25.998391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea23c3-7805-484d-93a3-ea8571729f32 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:26.052492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:26.053561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:26.053765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7381e0-9747-4417-ab73-7be114f88d51 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:26.054254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:26.054826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/94146067898240bfb358cef661f9b970/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:26.058874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, user_id=94146067898240bfb358cef661f9b970, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:26.060660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:26.095500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:26.096234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:26.096528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:26.096786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:26.097442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:26.099085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1cbc5-5b87-47d5-8089-501cf7aa30a3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:26.133124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 39/84] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:25 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/94146067898240bfb358cef661f9b970/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:26.133930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:26.137558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:26.200002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 46/85] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:25 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:26.259890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6131-642b-4c78-a28a-86c947805961 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:26.309433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:26.310542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:26.311057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:26.311529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] PATH_INFO: `/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1/users/aeec84e937e04e0cbd930c7b84cb139d/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:26.315156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=289b8d43f2774ba08aeee54b5e9f0bf1, user_id=aeec84e937e04e0cbd930c7b84cb139d, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:26.316917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac02f4-0cc7-4461-9080-dd5ef798cdc2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:26.383886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 40/86] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:26 2022] PUT /identity/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1/users/aeec84e937e04e0cbd930c7b84cb139d/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:28.233566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdee93-a180-44d5-9614-3df0de6ecc85 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.234068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdee93-a180-44d5-9614-3df0de6ecc85 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.234142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdee93-a180-44d5-9614-3df0de6ecc85 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.235479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 47/87] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:28.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cd067-edeb-427a-a696-03256b444894 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.249019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cd067-edeb-427a-a696-03256b444894 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.249500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cd067-edeb-427a-a696-03256b444894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.274341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6d3cd067-edeb-427a-a696-03256b444894 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:28.275495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6d3cd067-edeb-427a-a696-03256b444894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:28.349982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:28.359163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 41/88] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:28.545143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972969af-90f3-43bc-aaa5-789ee1603fc2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.545469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972969af-90f3-43bc-aaa5-789ee1603fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.545648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972969af-90f3-43bc-aaa5-789ee1603fc2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.546978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 48/89] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:28.555951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547751-7026-4608-ab31-37459705f40f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.556484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547751-7026-4608-ab31-37459705f40f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.556963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547751-7026-4608-ab31-37459705f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.579022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-17547751-7026-4608-ab31-37459705f40f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:28.580689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-17547751-7026-4608-ab31-37459705f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:28.674899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 42/90] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:28.878847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972969af-90f3-43bc-aaa5-789ee1603fc2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:28.904181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f88c-de26-4c48-8a62-5db8702512f2 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.904871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f88c-de26-4c48-8a62-5db8702512f2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.905347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f88c-de26-4c48-8a62-5db8702512f2 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.907214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 43/91] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:28.920613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:28.921181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:28.921657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:28.953017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:28.954576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:28.965782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.052173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972969af-90f3-43bc-aaa5-789ee1603fc2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.114640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 44/92] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:29.132237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.132824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.133110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.133384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.150226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:29.155651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:29.156514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:29.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 49/93] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:29.163061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.228566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.253609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.347859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e71d38-2284-4391-9a9e-a379770c7997 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.353224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29410977-6b43-432b-b9bf-c0ed820d0ea0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.380375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.395047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.395047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.395047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.395047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.401590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-97703151-a9a2-4799-a92d-45615b0e56b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:29.407398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:29.408263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:29.409021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:29.411262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 50/94] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:29.412671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e63a43-9178-4a38-a992-d0da4a342ef3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.413166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e63a43-9178-4a38-a992-d0da4a342ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.413637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e63a43-9178-4a38-a992-d0da4a342ef3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.415125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 51/95] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:29.417852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e63a43-9178-4a38-a992-d0da4a342ef3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.428069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.428961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.429422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.429879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.430828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:29.431374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:29.433626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8df07-2254-47ed-aec0-c737b49b9b64 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:29.446457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 45/96] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:29.448535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.449003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.449744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.472192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:29.474038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:29.533211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e63a43-9178-4a38-a992-d0da4a342ef3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.554921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 46/97] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:29.555909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.586027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78404463-229c-4715-8380-3d282f114eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.587101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404463-229c-4715-8380-3d282f114eec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.587529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404463-229c-4715-8380-3d282f114eec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.587987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404463-229c-4715-8380-3d282f114eec None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.588672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404463-229c-4715-8380-3d282f114eec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:29.589514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404463-229c-4715-8380-3d282f114eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:29.591139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404463-229c-4715-8380-3d282f114eec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:29.610874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.613587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 52/98] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:29.614399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78404463-229c-4715-8380-3d282f114eec None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.685971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925a790-dfa2-47fb-b51d-e9b53811d10b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.703168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.763564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78404463-229c-4715-8380-3d282f114eec None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.763871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.764444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.764738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.765021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.789527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:29.795217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:29.796086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:29.796852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 47/99] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:29.797487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.812355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.814821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.815367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.815916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.819631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:29.825190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:29.829271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 53/100] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:29.831336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.935430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8068-79e8-4912-beb1-6fbc1eff3482 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.945242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.956618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36fe4-1171-466b-90e3-8c5009684058 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:29.971779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:29.976060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.976702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:29.976985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:29.977263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:29.978406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:29.980043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:29.981987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 48/101] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:29.985453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:29.998686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:29.999297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.002246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.002246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.002246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:30.002246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.002994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.015287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 54/102] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:30.020710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.102874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56753dd9-3c8c-4cfd-b843-5f227838cb96 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.126318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9db9-c00d-4b44-b88a-c26e79c7e95e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.141280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.141951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.142236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.142544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.155718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-56ad35e5-8a4e-4e09-bc69-cb00bb861f28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:30.160543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:30.161415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:30.162202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 49/103] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:30.162882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccf0e-fae5-422d-9089-e8bce28f1637 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.163669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccf0e-fae5-422d-9089-e8bce28f1637 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.163669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ccf0e-fae5-422d-9089-e8bce28f1637 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.166623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 50/104] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:30.166623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ccf0e-fae5-422d-9089-e8bce28f1637 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.173234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.173853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.174138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.174412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.205340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.220435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.244534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.246531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.274536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ccf0e-fae5-422d-9089-e8bce28f1637 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.307236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.308335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.308615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.308887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.342534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 55/105] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:30.343486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.380278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.381764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d2a8b-31a0-4509-bcd2-420ba9475cf0 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.413549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.414476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.438369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5831404-b88c-440c-a507-22e51af23e4c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.470955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.471523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.471807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.472086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] PATH_INFO: `/v3/users/94146067898240bfb358cef661f9b970` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.473208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] RBAC: Authorizing `identity:get_user(user_id=94146067898240bfb358cef661f9b970)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.474690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.476833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 56/106] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/users/94146067898240bfb358cef661f9b970 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:30.477616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.483666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 51/107] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:30.483666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9074-312c-4b4a-89de-4b31770982ff None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.483666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9074-312c-4b4a-89de-4b31770982ff None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.483666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9074-312c-4b4a-89de-4b31770982ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.499105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ce7b9074-312c-4b4a-89de-4b31770982ff None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:30.500287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ce7b9074-312c-4b4a-89de-4b31770982ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:30.564589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 52/108] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:30.566029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574b68-8e48-49d9-9e05-c40fa5df32d2 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.566320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574b68-8e48-49d9-9e05-c40fa5df32d2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.566628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574b68-8e48-49d9-9e05-c40fa5df32d2 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.567878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 53/109] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:30.568585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574b68-8e48-49d9-9e05-c40fa5df32d2 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.586048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3702bfb-369d-4b27-bd29-d51e33f121b2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.620031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.621350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.621619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.621888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.622588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:30.622882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.624373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.633120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 57/110] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:30.633909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.638209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.640647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.678220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574b68-8e48-49d9-9e05-c40fa5df32d2 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.714061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.714723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.715012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.715287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.716427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.717878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.724438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 54/111] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:30.724438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.743393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee4ec0-06f1-44f8-9552-9af1fc94bf7f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.779082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d441067-00a2-4017-9633-649514692848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.779687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d441067-00a2-4017-9633-649514692848 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.780132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d441067-00a2-4017-9633-649514692848 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.780413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d441067-00a2-4017-9633-649514692848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.825255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e9ad-2e57-4a79-985f-beb5328a9cac None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:30.850364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d441067-00a2-4017-9633-649514692848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.851872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d441067-00a2-4017-9633-649514692848 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.853539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.855318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:30.858912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:30.859545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.859829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.860105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.861279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:30.862784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899b4ed-4939-4dfe-b99a-289c1bef17ac None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:30.868987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 55/112] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:30.868987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:30.868987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:30.868987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:30.884829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:30.886211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:30.950090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 58/113] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:30.950968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d441067-00a2-4017-9633-649514692848 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:30.956039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 56/114] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:30.956874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.066303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d441067-00a2-4017-9633-649514692848 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.069216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.070930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b9dd0-e480-4bd7-a747-0b3c9bf0d576 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.100721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.105942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.106637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.106884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.107158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] PATH_INFO: `/v3/users/aeec84e937e04e0cbd930c7b84cb139d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.107719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.108345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] RBAC: Authorizing `identity:get_user(user_id=aeec84e937e04e0cbd930c7b84cb139d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.116495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.116495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.116495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.116708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.116775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] RBAC: Adding query filter params (role.id=5eaea2af31a1491a9e7ad2736d630ee6, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd, user.id=aeec84e937e04e0cbd930c7b84cb139d) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:31.116775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.116928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 59/115] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/users/aeec84e937e04e0cbd930c7b84cb139d => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.116994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.117057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.121898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 57/116] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:30 2022] GET /identity/v3/role_assignments?user.id=aeec84e937e04e0cbd930c7b84cb139d&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.122797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.248966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a230d-adf3-4c02-ac42-3b3a71877584 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.250227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c134-4a89-4c19-b536-c4ca41e44b25 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.285934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.286742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.286869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.286869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.286997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.287064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.287138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.287347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.287829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] RBAC: Adding query filter params (role.id=2aaa59501e4146f5adce6e3872912797, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd, user.id=aeec84e937e04e0cbd930c7b84cb139d) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:31.288077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.289571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.296801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 58/117] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/role_assignments?user.id=aeec84e937e04e0cbd930c7b84cb139d&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=2aaa59501e4146f5adce6e3872912797 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.299533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.303035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:31.308201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:31.309217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:31.310009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 60/118] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:31.311021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.355333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.390356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.443442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac1ae9-d282-4a79-8e6f-f5a8cace313e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.446190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cbd89-d1e7-4477-ac0a-ccef03561ae2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.482692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.482692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.482995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.483207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.488642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.489285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.489565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.489840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.491016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.492485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.494277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 59/119] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.495345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.560850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.562333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.592508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.598365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95ef-f97a-4f3a-961b-d1cabd30f018 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.622919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.631762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.632427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.632713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.632993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.634102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.635588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.641747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 60/120] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.641747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.662622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 61/121] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:31.663475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.741804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec97461-a59d-4344-b0cf-73ed8dfbf781 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.762036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41b84-12be-4c68-b2b4-6dae02944633 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.774942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.775559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.775848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.776132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.776842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:31.777161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.778564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.786897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 61/122] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.787675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.790325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.796138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.796752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.797043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.797322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.801309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd, user.id=94146067898240bfb358cef661f9b970) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:31.801309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.801309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.807354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 62/123] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/role_assignments?user.id=94146067898240bfb358cef661f9b970&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.811503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:31.851890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:31.901981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9c2fc-5cce-440b-9167-a40b2651c1c4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.940089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.940719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.941012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.941318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.941991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.943496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.944960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ddf9c-f972-4633-b981-b639b565e432 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:31.982058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:31.982729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:31.983013 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:31.983296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] PATH_INFO: `/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:31.984425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] RBAC: Authorizing `identity:get_project(project_id=289b8d43f2774ba08aeee54b5e9f0bf1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:31.985936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:31.987582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 63/124] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/projects/289b8d43f2774ba08aeee54b5e9f0bf1 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:31.988350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:32.023638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 62/125] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:16:31 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:32.025318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:32.069301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:32.121095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f011a5-4083-44b9-aa38-23cb69d7eda9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:32.147711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29cec-8c1f-4951-9c59-19c6836220d5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:32.159344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:32.160827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:32.161501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:32.162169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:32.162463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] PATH_INFO: `/v3/users/12bfc564f4374bcaa9a3fb0b5df4aa22` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:32.180782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] RBAC: Authorizing `identity:get_user(user_id=12bfc564f4374bcaa9a3fb0b5df4aa22)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:32.182572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:32.185070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 64/126] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:31 2022] GET /identity/v3/users/12bfc564f4374bcaa9a3fb0b5df4aa22 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:32.190723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:32.194464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:32.195119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:32.195409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:32.195690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:32.196869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] RBAC: Adding query filter params (role.id=5eaea2af31a1491a9e7ad2736d630ee6, scope.project.id=289b8d43f2774ba08aeee54b5e9f0bf1, user.id=aeec84e937e04e0cbd930c7b84cb139d) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:32.196869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:32.198220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:32.204614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 63/127] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:32 2022] GET /identity/v3/role_assignments?user.id=aeec84e937e04e0cbd930c7b84cb139d&scope.project.id=289b8d43f2774ba08aeee54b5e9f0bf1&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:32.308293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901859d0-8df9-4ee3-be41-b0a7bdc7a36e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:32.333332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:32.346041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:32.346739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:32.347028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:32.347308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:32.348491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:32.350003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa6ec-d70c-4da5-9ca5-a7e0a1316941 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:32.351869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 65/128] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:32 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:32.358126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:32.463883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14fce9-0dbf-4c2d-ab37-6838a17ac082 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:32.498010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:32.498703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:32.498996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:32.499278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/12bfc564f4374bcaa9a3fb0b5df4aa22/roles/09b2cc11dfd1423cb451f90ab0447488` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:32.501196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, user_id=12bfc564f4374bcaa9a3fb0b5df4aa22, role_id=09b2cc11dfd1423cb451f90ab0447488)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:32.502704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9edfd-31c0-4b5e-8043-c404776e5a8f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:32.589886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 64/129] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:32 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/users/12bfc564f4374bcaa9a3fb0b5df4aa22/roles/09b2cc11dfd1423cb451f90ab0447488 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:34.224381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b169b-42ae-4387-bf80-6f317caf48f1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.224923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b169b-42ae-4387-bf80-6f317caf48f1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.224923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b169b-42ae-4387-bf80-6f317caf48f1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.226290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 66/130] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:34.232354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9cf7-27e9-44b9-b4d0-06ea5baac5d7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.232753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9cf7-27e9-44b9-b4d0-06ea5baac5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.233056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d9cf7-27e9-44b9-b4d0-06ea5baac5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.251030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3f2d9cf7-27e9-44b9-b4d0-06ea5baac5d7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:34.252224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3f2d9cf7-27e9-44b9-b4d0-06ea5baac5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:34.277394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:34.318075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 65/131] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:34.586777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b169b-42ae-4387-bf80-6f317caf48f1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:34.691068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b169b-42ae-4387-bf80-6f317caf48f1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:34.703154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fbc9-08af-4a4a-b1e7-bb882b1f6e5e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.703451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fbc9-08af-4a4a-b1e7-bb882b1f6e5e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.703742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559fbc9-08af-4a4a-b1e7-bb882b1f6e5e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.705102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 66/132] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:34.710410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.710726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.711007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.727352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:34.727987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.728274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.728551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.729003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:34.729696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:34.730217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:34.731201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:34.732929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 67/133] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:34.738225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:34.795237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:34.797590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 67/134] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:34.835442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99711ec0-c042-442e-8387-5c9b7522e3ee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:34.868913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:34.869461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:34.869742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:34.870041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:34.870729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:34.872174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:34.957573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 68/135] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:34 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:16:35.066940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:35.163656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:35.172093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4d2db-d380-4081-91a0-5f1ba1a626e3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:35.205107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:35.205730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:35.206222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:35.206715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:35.218119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:35.223037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:35.223893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:35.224851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 68/136] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:35.230319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:35.340754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df9e2b-b398-4401-aae1-2cf857bc16c0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:35.357022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:35.378827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:35.379814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:35.380297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:35.380767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:35.381654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:35.382152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:35.383779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:35.394647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 69/137] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:35.400901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:35.519919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1a610-93fa-4eeb-bfa2-66dcb0062274 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:35.555954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:35.556601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:35.556921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:35.557216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] PATH_INFO: `/v3/users/12bfc564f4374bcaa9a3fb0b5df4aa22` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:35.558943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] RBAC: Authorizing `identity:get_user(user_id=12bfc564f4374bcaa9a3fb0b5df4aa22)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:35.560478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:35.562082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 69/138] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:35 2022] GET /identity/v3/users/12bfc564f4374bcaa9a3fb0b5df4aa22 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:35.567792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:35.672770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4a44c-a342-433b-acbc-430652c7b3bd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:35.707429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:35.708057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:35.708344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:35.708626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:35.756186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:35.779577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:35.781200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:35.888503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 70/139] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:35.894241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:35.917774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:36.006545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a4a-8ef7-4424-8e21-403feaf5db00 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:36.045635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b58044-7409-420f-b376-c2900571049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:36.046200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58044-7409-420f-b376-c2900571049d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:36.046485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58044-7409-420f-b376-c2900571049d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:36.046793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b58044-7409-420f-b376-c2900571049d None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:36.047930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b58044-7409-420f-b376-c2900571049d None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:36.049403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b58044-7409-420f-b376-c2900571049d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:36.051254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 70/140] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:35 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:36.057168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:36.139155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:36.163429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d84d3-0062-4b8b-b944-aa59cd262c87 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:36.201696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:36.202334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:36.202671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:36.202952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:36.203715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] RBAC: Adding query filter params (role.id=09b2cc11dfd1423cb451f90ab0447488, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd, user.id=12bfc564f4374bcaa9a3fb0b5df4aa22) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:36.204031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:36.205529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4738-a813-4203-9ad7-8f68e45f2e48 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:36.212564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 71/141] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:36 2022] GET /identity/v3/role_assignments?user.id=12bfc564f4374bcaa9a3fb0b5df4aa22&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=09b2cc11dfd1423cb451f90ab0447488 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:37.248133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a0dc-bf1a-40d3-b2d9-d08663746e93 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.248694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a0dc-bf1a-40d3-b2d9-d08663746e93 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.248694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92a0dc-bf1a-40d3-b2d9-d08663746e93 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.250002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 71/142] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:37.255291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd253-476c-40dd-a445-4e85573f87fc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.255702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd253-476c-40dd-a445-4e85573f87fc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.255790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd253-476c-40dd-a445-4e85573f87fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.274078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-20ffd253-476c-40dd-a445-4e85573f87fc None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:37.275359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20ffd253-476c-40dd-a445-4e85573f87fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:37.348945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 72/143] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:37.392602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c21ac-9e3b-42d3-b497-52efa2b4100a None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.392949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c21ac-9e3b-42d3-b497-52efa2b4100a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.393150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c21ac-9e3b-42d3-b497-52efa2b4100a None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.394495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 72/144] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:37.399726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141facb-a395-4615-b466-7b501c348602 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.400029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141facb-a395-4615-b466-7b501c348602 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.400318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141facb-a395-4615-b466-7b501c348602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.418885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0141facb-a395-4615-b466-7b501c348602 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:37.420077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0141facb-a395-4615-b466-7b501c348602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:37.471567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04cab-1818-4bef-93b6-6e10d92971b3 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.471960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04cab-1818-4bef-93b6-6e10d92971b3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.472307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e04cab-1818-4bef-93b6-6e10d92971b3 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.473730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 73/145] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:37.478854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.479146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.479466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.488660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:37.494255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 73/146] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:37.497839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:37.499074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:37.565647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 74/147] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:37.619325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141facb-a395-4615-b466-7b501c348602 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:37.716876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141facb-a395-4615-b466-7b501c348602 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:37.749852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:37.750570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.750856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.751184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.759826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:37.761890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:37.766780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:37.767856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:37.768764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 74/148] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:37.775223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:37.780632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:37.839661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:37.862571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01df723-497f-41b2-a31a-516fd3578678 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:37.879252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae6690-cfeb-4665-8cf8-cf8f1d53b8cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:37.897758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:37.898390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.898705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.898987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.909081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:37.912891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:37.913486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:37.913765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:37.913913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:37.914963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:37.915889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:37.916507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:37.916507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:37.916674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 75/149] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:16:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:37.916745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:37.916816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:37.916888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:37.925799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 75/150] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:37.926551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:37.994470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.018377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2769-cb81-49f0-adac-f0e15a78a622 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.026886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefcb0-df9f-478f-b4d4-8a93a6a21854 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.048700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.052623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.056422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.056422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.056422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.056422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.056422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.058588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 76/151] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:38.059182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.061412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.062084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.062475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.062782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.063444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:38.063745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.065102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.075547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 76/152] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:16:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:38.076637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.158475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaaf17-a8b4-4824-8f05-1b175f95cdef None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.174137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a054aad-a1c0-480e-936b-02889796fe1f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.192401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.193030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.193323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.193600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] PATH_INFO: `/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.206679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] RBAC: Authorizing `identity:get_user(user_id=d5f9cc1663be4a82ad38f2ebe7061b9b)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.213220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.213331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.213331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.213516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 77/153] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:38 2022] GET /identity/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:38.213619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.213619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.213877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.213964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.213964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.215314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.300318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 77/154] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:38 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:38.301268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.322669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8669987-bec5-42cf-a77b-ef774542945b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.349779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.364160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.365055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.365180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.365355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] PATH_INFO: `/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.366560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=d5f9cc1663be4a82ad38f2ebe7061b9b)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.368181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.370669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 78/155] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:38 2022] GET /identity/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:38.376197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.421731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1accfd8-ec01-4193-a3c3-9f252a4d536a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.456305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.460258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.460943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.461226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.461502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.462708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.464295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.466038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 78/156] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:38 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:38.471611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.492736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc897e2-db6b-4f54-9c6f-2a3688d0a3b3 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.530828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.531467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.531751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.532025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.533754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.535244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.537071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 79/157] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:38 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:38.543693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:38.599739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43a3ab-3a63-4b45-9463-ae7bd0750459 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.612111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.642120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.643194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.643475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.643748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/d5f9cc1663be4a82ad38f2ebe7061b9b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.645798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, user_id=d5f9cc1663be4a82ad38f2ebe7061b9b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.647274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2c63d-7b5e-4eba-a47a-629f6d5a15d6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.658687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478ac4-281c-44bf-8bed-8fd771ee2bdf None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:38.694911 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:38.695508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:38.695793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:38.696072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/d5f9cc1663be4a82ad38f2ebe7061b9b/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:38.698310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, user_id=d5f9cc1663be4a82ad38f2ebe7061b9b, role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:38.699844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf94c8-19bf-4252-986a-12eb3254c45b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:38.712681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 79/158] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:38 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/d5f9cc1663be4a82ad38f2ebe7061b9b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:16:38.772318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:38.777974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 80/159] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:38 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/d5f9cc1663be4a82ad38f2ebe7061b9b/roles/2aaa59501e4146f5adce6e3872912797 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:16:40.570793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d32e27-74cb-4fab-99ca-a281892684af None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.571324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d32e27-74cb-4fab-99ca-a281892684af None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.571324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d32e27-74cb-4fab-99ca-a281892684af None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.573468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 80/160] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:40.579188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4662c-ff1a-4413-909c-aff415dfc543 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.579707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4662c-ff1a-4413-909c-aff415dfc543 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.579806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4662c-ff1a-4413-909c-aff415dfc543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.599906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-89e4662c-ff1a-4413-909c-aff415dfc543 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:40.601142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-89e4662c-ff1a-4413-909c-aff415dfc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:40.671305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 81/161] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:40.704978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5da9-b1a5-4a39-8575-8075d30af3e7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.705285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5da9-b1a5-4a39-8575-8075d30af3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.705533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d5da9-b1a5-4a39-8575-8075d30af3e7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.706921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 81/162] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:40.712484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df030ce4-6d77-439d-8113-b5372b0033f3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.712784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df030ce4-6d77-439d-8113-b5372b0033f3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.713066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df030ce4-6d77-439d-8113-b5372b0033f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.732190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-df030ce4-6d77-439d-8113-b5372b0033f3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:40.733385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-df030ce4-6d77-439d-8113-b5372b0033f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:40.804119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 82/163] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:40.913584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25cf9-025d-4e3a-a3be-5b41d8a5f67c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.913876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25cf9-025d-4e3a-a3be-5b41d8a5f67c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.914075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc25cf9-025d-4e3a-a3be-5b41d8a5f67c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.915446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 82/164] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:40.921005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5024f83-7e68-4ca4-86b0-0546540f48bd None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:40.921293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5024f83-7e68-4ca4-86b0-0546540f48bd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:40.921569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5024f83-7e68-4ca4-86b0-0546540f48bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:40.940632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e5024f83-7e68-4ca4-86b0-0546540f48bd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:40.941803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e5024f83-7e68-4ca4-86b0-0546540f48bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:40.942977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc25cf9-025d-4e3a-a3be-5b41d8a5f67c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.005823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.021340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 83/165] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:41.022623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573a08-d738-4935-bd26-412b24707003 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.022916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573a08-d738-4935-bd26-412b24707003 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.028998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee573a08-d738-4935-bd26-412b24707003 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.028998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 84/166] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:41.032406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.033370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.034135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.059116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc25cf9-025d-4e3a-a3be-5b41d8a5f67c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.059368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:41.060719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:41.098525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.099263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.099546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.099869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.111870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:41.117483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:41.118444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:41.119312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 83/167] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:41.120546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.145791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 85/168] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:41.147244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.155614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.229039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015a9e3-81d9-4735-8673-de840395d3c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.270445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.270658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f6810-15b9-432b-b5c3-809bc587aad1 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.271521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.272019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.272657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.274076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.275832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.277724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 84/169] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:41.283834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.294179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.313166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.314098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.314682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.314968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.315802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:41.316128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.318162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.332387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 86/170] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:41.332387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.382392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.389586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f8ff-ef78-48db-8c3c-a91c94a1d9b8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.424667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.425809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.426225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.426517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.427175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.428625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.443519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6193c-9c95-44ca-98f9-d0015a5c53df None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.475736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.476359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.476640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.476908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:41.492694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:41.493517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:41.495500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 87/171] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:41.496313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.523574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 85/172] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:41 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:41.524417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.537489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.600631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5452bb-a77a-4bae-899b-5896a0517162 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.637065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.637711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.637993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.638270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] PATH_INFO: `/v3/users/746f00d37ea347c5944cb923885c156f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.644880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd046a0b-aa25-4c21-b786-a129f3459842 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.655059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] RBAC: Authorizing `identity:get_user(user_id=746f00d37ea347c5944cb923885c156f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.656540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.658261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 88/173] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/users/746f00d37ea347c5944cb923885c156f => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:41.658261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.688687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.689326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.689604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.689874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.703356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:41.707025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:41.707984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:41.708787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 86/174] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:41.709437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.731369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.779189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:41.780568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6317fa-07da-403c-b4de-64f822f30c99 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.821351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.821972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.822255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.822607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.823312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:41.823617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.825002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.829378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7838b-dbbb-4aeb-8c0e-8a7bb9f0c845 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.835162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 89/175] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:41.836004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.872910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:41.873937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:41.874427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:41.874949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:41.876264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:41.879566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:41.881556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 87/176] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:41.882322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:41.962423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd5347-238b-45f8-b58b-4b333502b1d6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:41.986937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.003476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122e28-0e0f-452c-927e-7c53f434e5ca None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.016206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.017256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.017540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.017814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.019017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:42.019640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.021231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.033396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 90/177] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:42.034430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.043094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.043998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.044356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.044723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] PATH_INFO: `/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.045941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] RBAC: Authorizing `identity:get_user(user_id=d5f9cc1663be4a82ad38f2ebe7061b9b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.047502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.049347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 88/178] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:41 2022] GET /identity/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:42.049992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.067337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.163891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299aa5f-90ba-4b8d-9e2f-50526e3fb70d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.165694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1355d-66f0-40d6-9b39-93332f034519 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.202337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.203279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.203610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.203888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/746f00d37ea347c5944cb923885c156f/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.205982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, user_id=746f00d37ea347c5944cb923885c156f, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.206406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.207154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.207501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.207614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.207867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] PATH_INFO: `/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.209055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=d5f9cc1663be4a82ad38f2ebe7061b9b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.211603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.214251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 89/179] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/users/d5f9cc1663be4a82ad38f2ebe7061b9b => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:42.215081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.237111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.271787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 91/180] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:42 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/746f00d37ea347c5944cb923885c156f/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:42.273282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.378890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75431ff-0b55-47c6-9104-cb81986bbbc0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.418306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.452315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.452941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.453221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.453457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fff93-40ee-4283-a8bc-66a6ae0eb0fa None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.453564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.531570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.533384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.533663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.533933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.570305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.572003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.602023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.636693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.642532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.730527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 90/181] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:42.736056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.841946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9218-6b18-4a27-a438-b4fb95598674 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:42.867111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.879228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 92/182] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:42.884648 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:42.888560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:42.899403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:42.900057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:42.900339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:42.900618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:42.901792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:42.903285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:42.904905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 91/183] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:42.918074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:43.068001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb26de-be5d-4486-92c9-fc51144bbe54 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:43.085783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7b127-1e53-450c-84dc-a406d25c4ec8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:43.138770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:43.139694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:43.140152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:43.140600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:43.141727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=c400356aae10410e9cfef1cc92dac4ae, user.id=d5f9cc1663be4a82ad38f2ebe7061b9b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:43.142197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:43.142309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:43.142928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:43.143267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:43.143547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:43.144414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d22b31-6487-4007-90ca-3dea406070ce None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:43.144666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:43.154387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:43.154387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 93/184] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:43.154387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:43.159378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 92/185] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:42 2022] GET /identity/v3/role_assignments?user.id=d5f9cc1663be4a82ad38f2ebe7061b9b&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:43.220795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:43.279062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466ee8-6315-4673-8cb5-9806e1fd61c6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:43.336386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:43.337020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:43.337301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:43.337574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:43.338325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] RBAC: Adding query filter params (role.id=2aaa59501e4146f5adce6e3872912797, scope.project.id=c400356aae10410e9cfef1cc92dac4ae, user.id=d5f9cc1663be4a82ad38f2ebe7061b9b) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:43.339190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:43.340703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764885c1-95b1-4c35-8e78-8eff5c1e0e93 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:43.348204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 94/186] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:43 2022] GET /identity/v3/role_assignments?user.id=d5f9cc1663be4a82ad38f2ebe7061b9b&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=2aaa59501e4146f5adce6e3872912797 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:43.974392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b0701-40c9-4935-b1a0-339dcb385003 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:43.975453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b0701-40c9-4935-b1a0-339dcb385003 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:43.975942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b0701-40c9-4935-b1a0-339dcb385003 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:43.977604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 93/187] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:43.983466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:43.984365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:43.984838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.003897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:44.004714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:44.064044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7063-0c87-4ef5-8e75-f9421392929c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.064345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7063-0c87-4ef5-8e75-f9421392929c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.064603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d7063-0c87-4ef5-8e75-f9421392929c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.066059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 94/188] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:44.070893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.071177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.071452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.076075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 95/189] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:44.123205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:44.124387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:44.167372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:44.189926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 95/190] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:44.343023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.390171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:44.443016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00a852-6309-4ef4-b8db-be60d39c6966 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.454408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.477703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.478341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.478662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.478998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.489802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:44.494752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:44.496183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:44.496977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 96/191] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:44.502050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.572254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d71fb-98a2-4ad4-815c-1bfcd3b6a429 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.609702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.610605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.610888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.611192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.611981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a989ad-1b29-4048-aec3-c3ca8d8ec06c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.612444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:44.614163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:44.617004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 96/192] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:44.620642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:44.623110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.654454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.655100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.655374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.655651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.656326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:44.656629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:44.658025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:44.668565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 97/193] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:44.673596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.678829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:44.739827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c743e-7c7e-448c-a946-851e7b4b48ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.772036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.772643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.772923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.773086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614b39-a2b6-47e3-8052-ecdba0c5af7e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.773184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.773821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:44.775300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4f677-47eb-4bec-b077-c360e0ce27a6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:44.806899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.807561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.807860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.808111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] PATH_INFO: `/v3/users/7b22a355484e4be491abe25c48070b60` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.820736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] RBAC: Authorizing `identity:get_user(user_id=7b22a355484e4be491abe25c48070b60)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:44.822177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea45d55-9b1b-40e3-914a-7e5866226d4b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:44.823819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 98/194] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:44 2022] GET /identity/v3/users/7b22a355484e4be491abe25c48070b60 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:44.825075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395f08-0263-4bf2-bb7b-bce317e3ba55 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.830421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395f08-0263-4bf2-bb7b-bce317e3ba55 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.830421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27395f08-0263-4bf2-bb7b-bce317e3ba55 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.830421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 99/195] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:44.830421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27395f08-0263-4bf2-bb7b-bce317e3ba55 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:44.855852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:44.877916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 97/196] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:44 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:16:44.879133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.879423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.879696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.900321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:44.902358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:44.936818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27395f08-0263-4bf2-bb7b-bce317e3ba55 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:44.976152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 98/197] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:44.977034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:44.977944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:44.978367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:44.978819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:44.980276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:44.981982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:44.983942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 100/198] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:44 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:44.990786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:45.033736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:45.107897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabb59-d402-4d91-b1ac-ad3e9ab9ff20 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:45.145713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:45.146311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:45.146686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:45.146968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/7b22a355484e4be491abe25c48070b60/roles/09b2cc11dfd1423cb451f90ab0447488` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:45.148882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, user_id=7b22a355484e4be491abe25c48070b60, role_id=09b2cc11dfd1423cb451f90ab0447488)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:45.150323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:45.219522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 99/199] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:16:44 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/users/7b22a355484e4be491abe25c48070b60/roles/09b2cc11dfd1423cb451f90ab0447488 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:45.247905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:45.360393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:45.366747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee154a-f2d0-42f2-8378-48de04e1cf90 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:45.404762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:45.405396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:45.405975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:45.406249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:45.416943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:45.421795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:45.422712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:45.423522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 101/200] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:45.428741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:45.531704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:45.540695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362044e-c00e-4e3a-b4bf-48a3196d6e94 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:45.578709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:45.579332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:45.579616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:45.579895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:45.580569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:45.580876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:45.582263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:45.591774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 100/201] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:45.597323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:45.713134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a129a-c5fb-410c-a17c-d6e89f938e9a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:45.752508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:45.753160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:45.753444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:45.753720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] PATH_INFO: `/v3/users/746f00d37ea347c5944cb923885c156f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:45.754901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] RBAC: Authorizing `identity:get_user(user_id=746f00d37ea347c5944cb923885c156f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:45.756371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:45.758276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 102/202] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:45 2022] GET /identity/v3/users/746f00d37ea347c5944cb923885c156f => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:45.763887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:45.866602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dae66-9219-4104-bea2-a9c4baa94fda None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:45.900805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:45.901763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:45.902238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:45.902745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:45.928187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:45.975170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:45.976550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:46.079344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 101/203] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:46.085253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:46.111821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:46.188201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f68bd-02c6-423a-98d5-7dad36c08554 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:46.226278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:46.226874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:46.227159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:46.227440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:46.228580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:46.230598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4f01f-040a-44c9-9ab2-2ea465e24169 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:46.232421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 103/204] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:46 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:46.238110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:46.299292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:46.344266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bb87f-ff4a-481d-a06c-be82474cc75b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:46.378589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:46.379238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:46.379473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:46.379751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:46.380478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] RBAC: Adding query filter params (role.id=5eaea2af31a1491a9e7ad2736d630ee6, scope.project.id=c400356aae10410e9cfef1cc92dac4ae, user.id=746f00d37ea347c5944cb923885c156f) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:46.380785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:46.382230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c037b00-50a7-44dc-afd7-c6e65220a8ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:46.388986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 102/205] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:46 2022] GET /identity/v3/role_assignments?user.id=746f00d37ea347c5944cb923885c156f&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:47.245796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8259d-c3ad-47c8-b11b-236cbb90c18c None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.246297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8259d-c3ad-47c8-b11b-236cbb90c18c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.246372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8259d-c3ad-47c8-b11b-236cbb90c18c None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.247872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 104/206] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:47.252864 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be26ce5-db57-4db9-b1f2-f6286cb6a01f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.253315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be26ce5-db57-4db9-b1f2-f6286cb6a01f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.253405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be26ce5-db57-4db9-b1f2-f6286cb6a01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.271463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2be26ce5-db57-4db9-b1f2-f6286cb6a01f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:47.272645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2be26ce5-db57-4db9-b1f2-f6286cb6a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:47.316844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bac6f-990b-4fea-8aee-736f7d10f772 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.317131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bac6f-990b-4fea-8aee-736f7d10f772 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.317408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bac6f-990b-4fea-8aee-736f7d10f772 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.318935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 105/207] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:47.323699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166dee4-76b5-418a-ab50-af12012c3c7e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.323988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166dee4-76b5-418a-ab50-af12012c3c7e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.324271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166dee4-76b5-418a-ab50-af12012c3c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.341765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 103/208] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:47.343202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c166dee4-76b5-418a-ab50-af12012c3c7e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:47.344492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c166dee4-76b5-418a-ab50-af12012c3c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:47.419710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 106/209] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:47.483149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc0a5-42e2-4c25-9bd7-6085dc85dc56 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.483464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc0a5-42e2-4c25-9bd7-6085dc85dc56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.483712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc0a5-42e2-4c25-9bd7-6085dc85dc56 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.485189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 104/210] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:47.490259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.490571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.490845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.509370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:47.510495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:47.524346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:47.576343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 107/211] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:47.610552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc0a5-42e2-4c25-9bd7-6085dc85dc56 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:47.660116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:47.692359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:47.714412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc0a5-42e2-4c25-9bd7-6085dc85dc56 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:47.752777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:47.753414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.753698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.753974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.764122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:47.768890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:47.770010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:47.771036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 105/212] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:47.778172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:47.803580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140858-2a5f-4533-acdd-cf9bb157062a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:47.839345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:47.839991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.840275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.840553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.841672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:47.843165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:47.844792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 108/213] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:47.845550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:47.864380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:47.886635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cdc33-3ff3-4227-92ee-84e5968d0245 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:47.896223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:47.920915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:47.921516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.921791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.922063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.922761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:47.923323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:47.925420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:47.933659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 106/214] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:47.934396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:47.945545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3391-9ab8-422c-95c9-ab379f069e5d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:47.979926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:47.980571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:47.980843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:47.981117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:47.990987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:47.995775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:47.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 109/215] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:48.002694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.037859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ac81-8480-4b8a-906c-3d6339af7d45 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.069873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.070472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.070791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.071070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.071711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.073666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.080131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.100987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a70643-9840-48ff-8ee9-62426d3e079b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.134683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.135345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.135626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.135901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] PATH_INFO: `/v3/users/b7dddaa0324647a0b54d305ca803fd45` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.140256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 107/216] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:16:47 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:48.144034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.149919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] RBAC: Authorizing `identity:get_user(user_id=b7dddaa0324647a0b54d305ca803fd45)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.151654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.153615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 110/217] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:47 2022] GET /identity/v3/users/b7dddaa0324647a0b54d305ca803fd45 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:48.159576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.179870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.264900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55c83c-0263-458e-b387-75c6cbf4d49f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.269580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178dd19-7824-42c3-aee1-f2e9ab35ffcf None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.306227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.306625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.306873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.307205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.307305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.307483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.307577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.307752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.308199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:48.308488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] RBAC: Adding query filter params (role.id=5eaea2af31a1491a9e7ad2736d630ee6, user.id=b7dddaa0324647a0b54d305ca803fd45, scope.system=all) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:48.308633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.308796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.309881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.310217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.312852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.316236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 111/218] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/role_assignments?user.id=b7dddaa0324647a0b54d305ca803fd45&scope.system=all&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:48.320213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 108/219] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:48.326260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.447827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad2bb-214e-4fee-9d7e-de0f8c7bbffe None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.487504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.488157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.488434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.488735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] PATH_INFO: `/v3/users/7b22a355484e4be491abe25c48070b60` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.489874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] RBAC: Authorizing `identity:get_user(user_id=7b22a355484e4be491abe25c48070b60)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.491353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.492928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 112/220] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/users/7b22a355484e4be491abe25c48070b60 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:48.498807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.593878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.606328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b86b-4cbf-4c20-894a-4282d4d07389 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.643429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.644045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.644331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.644608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.716576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.718058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.814865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.819609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 109/221] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:16:48.825299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:48.899710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:48.933809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839790e3-0ffd-485f-bfc5-d67525e330b2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:48.969747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:48.970438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:48.970750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:48.971074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:48.972211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:48.973658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69861-fde8-45d7-9d89-f17feca826b5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:48.975333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 113/222] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:48.981346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:49.085883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04c207-96fb-40c5-ab67-cb919f063e85 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:49.121574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:49.122187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:49.122469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:49.122769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:49.123500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] RBAC: Adding query filter params (role.id=09b2cc11dfd1423cb451f90ab0447488, scope.project.id=c400356aae10410e9cfef1cc92dac4ae, user.id=7b22a355484e4be491abe25c48070b60) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:49.123808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:49.125250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e2c7-bcc4-4df8-8280-516537d67234 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:49.131405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 110/223] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:16:48 2022] GET /identity/v3/role_assignments?user.id=7b22a355484e4be491abe25c48070b60&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=09b2cc11dfd1423cb451f90ab0447488 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:50.493296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb033-733c-4e81-999c-e5b5ff126621 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.493848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb033-733c-4e81-999c-e5b5ff126621 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.493848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb033-733c-4e81-999c-e5b5ff126621 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.495906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 114/224] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:50.499606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc4d0-8e8e-47e4-911e-a1b2cb3eaf13 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.499991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc4d0-8e8e-47e4-911e-a1b2cb3eaf13 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.500152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14dc4d0-8e8e-47e4-911e-a1b2cb3eaf13 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.500458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f497b-8f49-44eb-b063-b6ed281e320c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.500748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f497b-8f49-44eb-b063-b6ed281e320c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.501022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f497b-8f49-44eb-b063-b6ed281e320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.501606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 111/225] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:50.506264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778381b-d4ec-4605-945f-716d11dbe62f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.506581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778381b-d4ec-4605-945f-716d11dbe62f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.506868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778381b-d4ec-4605-945f-716d11dbe62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.520693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-153f497b-8f49-44eb-b063-b6ed281e320c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:50.521867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-153f497b-8f49-44eb-b063-b6ed281e320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:50.525376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c778381b-d4ec-4605-945f-716d11dbe62f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:50.526599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c778381b-d4ec-4605-945f-716d11dbe62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:50.572461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:50.591217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 115/226] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:50.597837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 112/227] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:50.597975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807412a1-ed0c-49e0-9d7d-a437f271c115 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.597975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807412a1-ed0c-49e0-9d7d-a437f271c115 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.597975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807412a1-ed0c-49e0-9d7d-a437f271c115 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.597975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 116/228] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:50.603366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:50.603656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:50.603944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:50.622724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:50.623910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:50.690457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 113/229] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:50.858743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807412a1-ed0c-49e0-9d7d-a437f271c115 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:50.859842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:50.926605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:50.965707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807412a1-ed0c-49e0-9d7d-a437f271c115 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:50.967285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfe011-117f-4179-9952-649bb72df7ec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:50.980853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.002349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.003021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.003299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.003436 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.004002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.004074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.004260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.004460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.013942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:51.015501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:51.018781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:51.019680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:51.020492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 117/230] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:51.021607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:51.021674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:51.023469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:51.024466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 114/231] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:51.025816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.125551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7465-72bf-419c-9424-32d9d5520f8b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.126593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3dc23-59e8-4494-88f3-74567f0d3f1d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.150303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.158118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.164794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.164940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.164940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.165098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.165179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.165267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.165267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.165412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:51.165412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.165554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.165646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.165725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.165725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 118/232] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:51.165725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.172064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 115/233] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:51.172064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.209094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.264059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80863f-8e2e-4957-bf91-6b67e5a9b530 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.273504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca442ef-5db0-48c9-8356-6dc99542e84f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.297186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.297786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.298060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.298332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.299020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:51.299320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.300687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.307566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.308167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.308401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 119/234] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:51.308491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.308707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] PATH_INFO: `/v3/groups` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.309173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.309326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.310779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.367049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.402622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 116/235] 10.222.0.39 () {60 vars in 1134 bytes} [Wed Oct 19 13:16:51 2022] POST /identity/v3/groups => generated 229 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:16:51.402622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.409931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d8efb-0f56-4856-ab8e-c53dddcc9c2c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.446603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.447386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.447667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.447950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] PATH_INFO: `/v3/users/b7dddaa0324647a0b54d305ca803fd45` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.449292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] RBAC: Authorizing `identity:get_user(user_id=b7dddaa0324647a0b54d305ca803fd45)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.450817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.452538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 120/236] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/users/b7dddaa0324647a0b54d305ca803fd45 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:51.458489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.525274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.533029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e4ee-4660-40b6-96a5-9f95cf2ca67a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.573244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.574339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.574339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.574546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] PATH_INFO: `/v3/users/a6a02ba78ebb4ee9b938021d927aefb7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.575930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3c2c4-a646-4589-979b-007df616146c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.590944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] RBAC: Authorizing `identity:get_user(user_id=a6a02ba78ebb4ee9b938021d927aefb7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.592403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.594386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 117/237] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/users/a6a02ba78ebb4ee9b938021d927aefb7 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:51.599762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:51.604032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:51.618299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.618998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.619279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.619560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] PATH_INFO: `/v3/system/users/b7dddaa0324647a0b54d305ca803fd45/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.621130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b7dddaa0324647a0b54d305ca803fd45, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.622585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfb802-e94c-4fee-9096-70130cc4e999 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.687015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 121/238] 10.222.0.39 () {58 vars in 1334 bytes} [Wed Oct 19 13:16:51 2022] PUT /identity/v3/system/users/b7dddaa0324647a0b54d305ca803fd45/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:51.713392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d13c2-1934-4034-b35f-8555edf4e50e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:51.752871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:51.753502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:51.753786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:51.754063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:51.754844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] RBAC: Adding query filter params (role.id=09b2cc11dfd1423cb451f90ab0447488, user.id=a6a02ba78ebb4ee9b938021d927aefb7, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:51.755161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:51.756622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78920c37-70e3-479c-80a4-05061f9a4718 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:51.763209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 118/239] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Oct 19 13:16:51 2022] GET /identity/v3/role_assignments?user.id=a6a02ba78ebb4ee9b938021d927aefb7&scope.system=all&role.id=09b2cc11dfd1423cb451f90ab0447488 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:53.572204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c82e0f-5688-4a38-8c1a-5f89c081dbbc None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.572751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c82e0f-5688-4a38-8c1a-5f89c081dbbc None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.572751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c82e0f-5688-4a38-8c1a-5f89c081dbbc None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.574927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 122/240] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:53.580144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3543f-ba1e-489f-80c1-785699d03290 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.580542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3543f-ba1e-489f-80c1-785699d03290 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.580736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3543f-ba1e-489f-80c1-785699d03290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.599008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-79c3543f-ba1e-489f-80c1-785699d03290 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:53.600184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-79c3543f-ba1e-489f-80c1-785699d03290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:53.611331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:53.667065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 119/241] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:53.830901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43b7a-08cd-492a-a8ce-148a08191a7d None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.831086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43b7a-08cd-492a-a8ce-148a08191a7d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.831365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d43b7a-08cd-492a-a8ce-148a08191a7d None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.832842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 123/242] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:53.837981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.838269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.838564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.857401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:53.858602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:53.895836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099f9a-9c92-4346-b358-fb0ff6f21005 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.896141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099f9a-9c92-4346-b358-fb0ff6f21005 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.896393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d099f9a-9c92-4346-b358-fb0ff6f21005 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.897871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 124/243] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:53.902785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57329347-beae-47bd-b157-d2945b74e17c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:53.903067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57329347-beae-47bd-b157-d2945b74e17c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:53.903345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57329347-beae-47bd-b157-d2945b74e17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:53.921467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-57329347-beae-47bd-b157-d2945b74e17c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:53.922680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-57329347-beae-47bd-b157-d2945b74e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:53.929109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 120/244] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:53.938312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:53.992426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 125/245] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:16:54.025012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.047148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827e3f9-f716-44ea-8b44-1efaf90e649f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.081353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.081912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.082189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.082457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.083569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.085278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.087008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 121/246] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:54.093004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57329347-beae-47bd-b157-d2945b74e17c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.199394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.300010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57329347-beae-47bd-b157-d2945b74e17c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.318007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.335026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.335232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9f13-fc6c-4960-91d0-298aa83e640b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.335555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.335835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.336103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] PATH_INFO: `/v3/groups` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.336832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.338290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.368007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.368628 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.368906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.369175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.379327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:54.384526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:54.385595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 122/247] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:16:54.387180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.390132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.423048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 126/248] 10.222.0.39 () {60 vars in 1134 bytes} [Wed Oct 19 13:16:54 2022] POST /identity/v3/groups => generated 226 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:16:54.423872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.505305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926836b-c554-405e-84ac-697306fad5c8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.541301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dff8b6-8013-497a-9156-bc6fa3d761f4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.543664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.544327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.544612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.544892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.558108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:54.563759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:54.564782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:54.567480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 123/249] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:54.570862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.583942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.584878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.585145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.585361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.586063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:54.586366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.587782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.597943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 127/250] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:54.603620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.646721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.649133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.691224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc4d2d-b317-497c-bc3c-3796e307f1bb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.720716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037484-4d55-427b-9473-79c8c7464845 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.735769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.736417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.736702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.736976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.737669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:54.737954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.740524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.750571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 124/251] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:54.755255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.769484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.770766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.771170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.771348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] PATH_INFO: `/v3/users/b7dddaa0324647a0b54d305ca803fd45` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.773029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b7dddaa0324647a0b54d305ca803fd45)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.775101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.777008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 128/252] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/users/b7dddaa0324647a0b54d305ca803fd45 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:54.783473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.895719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c979573-57e6-43a9-819f-c2a3ee76780b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.916034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6917449-e3c0-49ac-9e39-2181388d32a4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:54.929910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.930477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:54.943038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.944096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.944440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.944943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] PATH_INFO: `/v3/users/a6a02ba78ebb4ee9b938021d927aefb7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.946705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] RBAC: Authorizing `identity:get_user(user_id=a6a02ba78ebb4ee9b938021d927aefb7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.948518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.951211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 125/253] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/users/a6a02ba78ebb4ee9b938021d927aefb7 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:54.957528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:54.959877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:54.960501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:54.960787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:54.961063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:54.961818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] RBAC: Adding query filter params (role.id=5eaea2af31a1491a9e7ad2736d630ee6, user.id=b7dddaa0324647a0b54d305ca803fd45, scope.system=all) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:54.962120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:54.964862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc92c-33cf-4be3-9be6-5f38088766f8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:54.980627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 129/254] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Oct 19 13:16:54 2022] GET /identity/v3/role_assignments?user.id=b7dddaa0324647a0b54d305ca803fd45&scope.system=all&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:55.092122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97093f67-8e8d-42fa-9561-6102373b4a7f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:55.137608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:55.138299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:55.138607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:55.138886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] PATH_INFO: `/v3/system/users/a6a02ba78ebb4ee9b938021d927aefb7/roles/09b2cc11dfd1423cb451f90ab0447488` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:55.140460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a6a02ba78ebb4ee9b938021d927aefb7, role_id=09b2cc11dfd1423cb451f90ab0447488)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:55.141881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ed0a8-6df6-41d4-9737-d8227432434e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:55.212798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 126/255] 10.222.0.39 () {58 vars in 1334 bytes} [Wed Oct 19 13:16:54 2022] PUT /identity/v3/system/users/a6a02ba78ebb4ee9b938021d927aefb7/roles/09b2cc11dfd1423cb451f90ab0447488 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:56.918056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c780-3524-4617-bc6d-76458df4fd7b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:56.918589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c780-3524-4617-bc6d-76458df4fd7b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:56.918668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2c780-3524-4617-bc6d-76458df4fd7b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:56.920016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 130/256] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:56.925657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:56.926132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:56.926211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:56.954066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:56.955363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:57.036909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 127/257] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 13:16:57.531857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2c780-3524-4617-bc6d-76458df4fd7b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:57.548119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2c780-3524-4617-bc6d-76458df4fd7b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:57.552939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:57.553798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.554251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.554723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.570158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-9c106430-644d-4ee1-8183-c27e86a1442e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:57.575955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:57.576866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:57.577675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:57.580016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 131/258] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:16:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:57.586044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:57.590298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de87b-9670-4673-81a5-7bc52e984003 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.590775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de87b-9670-4673-81a5-7bc52e984003 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.591193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de87b-9670-4673-81a5-7bc52e984003 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.593078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 132/259] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:57.595083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:57.599204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273388a-b850-4df2-93d4-b63fa7ab6b21 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:57.600165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679efd3-da71-411d-90a7-fe329de531da None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.600629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679efd3-da71-411d-90a7-fe329de531da None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.601136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679efd3-da71-411d-90a7-fe329de531da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.608184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:57.608830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.609107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.609385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:16:57.625934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:57.633159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-a679efd3-da71-411d-90a7-fe329de531da None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:57.633357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 128/260] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:16:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:57.634813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a679efd3-da71-411d-90a7-fe329de531da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:57.639014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:57.646034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 133/261] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 13:16:57.657025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d61b9-fca5-4910-b3d8-9ec929d11d45 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:57.666730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:57.667982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.668263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.668533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.681012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:57.682190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:57.682190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:16:57.682190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:16:57.683619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:57.685430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 129/262] 10.222.0.39 () {56 vars in 1201 bytes} [Wed Oct 19 13:16:57 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:57.691790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679efd3-da71-411d-90a7-fe329de531da None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:57.705057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679efd3-da71-411d-90a7-fe329de531da None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:57.712499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:57.713393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.714496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.714963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.718129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:57.720499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbc1b8-9c4d-481f-9aa0-9f3e5c1c1275 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:57.722324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 134/263] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:16:57 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:57.728169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:57.741839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec313-5ec4-4ae5-a581-ef4f82edc436 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:57.750492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:57.751201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.751480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.751755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/groups/ba53d755f118436fa61580bb69747fcc/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.753735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, group_id=ba53d755f118436fa61580bb69747fcc, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:57.754758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:57.754758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:16:57.754758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:16:57.756782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:57.844489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61364-98a5-4caf-ad64-ea52cff981d7 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.845121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61364-98a5-4caf-ad64-ea52cff981d7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.845623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61364-98a5-4caf-ad64-ea52cff981d7 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.847193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 135/264] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:57.848228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 130/265] 10.222.0.39 () {58 vars in 1463 bytes} [Wed Oct 19 13:16:57 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/groups/ba53d755f118436fa61580bb69747fcc/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:57.852531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:57.853086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:57.853533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:57.879966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:57.881451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:57.952464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 136/266] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 13:16:58.142429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.153635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd0c6-abfc-4783-bc9d-1e3bf787211b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.160658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.161321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.161606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.161885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.174428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-298114ba-a7e4-4e3b-8b46-3fc5d0e6af81 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:58.184168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:58.185127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:58.185927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 131/267] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:58.193343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.206718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cfcdd-f54f-4740-9a8d-49856fc1600a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.215104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.216063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.216539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.217006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.217934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:58.218439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.220433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:58.220433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:16:58.220433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:16:58.221688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.237859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 137/268] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:58.243188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebfa2c-3565-4083-b9de-d6896d3981d2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.243513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebfa2c-3565-4083-b9de-d6896d3981d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.243756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebfa2c-3565-4083-b9de-d6896d3981d2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.244545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.245480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 132/269] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:16:58.250336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.250655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.250938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.256101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b716-f4fd-4086-96fd-4483292ab4a7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.264525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.265462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.265963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.266434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.268637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.269862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:58.269862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:16:58.269862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:16:58.271500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.273562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:58.273781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 138/270] 10.222.0.39 () {56 vars in 1201 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:58.278210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:58.282796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.303678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355f9c-dc8a-414d-9f4b-53abb2a5a652 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.309397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.310274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.310782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.311289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.312597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.314918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.317069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 139/271] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:58.318045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.335353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4a9eb-9f3c-458a-b363-901049138398 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.347069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.348008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.348503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.349619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.373538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 133/272] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:16:58.374490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.383267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:58.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:58.390432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:16:58.391286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:58.394614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4ebee-e589-4390-9c8e-36fdc1605609 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.400286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 140/273] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 82 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:58.400960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.402258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.402564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.402906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/groups/ba53d755f118436fa61580bb69747fcc/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.405290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6026ce3c10a4e3f9da5807a93cd12bd, group_id=ba53d755f118436fa61580bb69747fcc, role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.412667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.419390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.438337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d74ede-2764-4ed6-9f08-e27614a6e9e8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.445212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.445867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.446144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.446417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.447176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:58.447475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.449632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.466408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 141/274] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:58.472017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.486882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 134/275] 10.222.0.39 () {58 vars in 1463 bytes} [Wed Oct 19 13:16:58 2022] PUT /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd/groups/ba53d755f118436fa61580bb69747fcc/roles/2aaa59501e4146f5adce6e3872912797 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:58.487875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96bede-208f-4d4c-a795-f51d21a1a2d5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.559161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:58.567401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.568111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.568458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.568761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.569892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.571898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.574469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 142/276] 10.222.0.39 () {56 vars in 1201 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:58.580065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.602563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8d05-58db-4831-a90a-e3ccad526667 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.610424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.611432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.611894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.612348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.613766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.616098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.618312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 135/277] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:58.625103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.639048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811be71e-c6d9-43b7-9abb-adf240b2e645 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.644681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.645379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.645661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.645933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/groups/ba53d755f118436fa61580bb69747fcc/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.649025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, group_id=ba53d755f118436fa61580bb69747fcc, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:58.650112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:16:58.650112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:16:58.650112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:16:58.652782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:58.727465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 143/278] 10.222.0.39 () {58 vars in 1463 bytes} [Wed Oct 19 13:16:58 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/groups/ba53d755f118436fa61580bb69747fcc/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:16:58.729126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:58.826912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:58.853626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610e153-b65b-4bbf-af15-5fb9edb66f12 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:58.893727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:58.894991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:58.898888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:58.899409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:58.915586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-11a10d12-de85-4b35-a426-20a54371cfbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:58.921742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:58.922759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 13:16:58.923568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:16:58.926290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 136/279] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:58.932104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.109412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f7dd9-fe1f-4022-9a2e-c4bc69a7defe admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.148235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6e23-4a86-4c22-be26-f802f5d8b269 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.149231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6e23-4a86-4c22-be26-f802f5d8b269 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.149855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b6e23-4a86-4c22-be26-f802f5d8b269 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.151052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.152235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.152519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.152776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 137/280] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:59.152876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.154538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:59.154538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.156406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.157880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.158162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.158431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.170679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 144/281] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:16:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:59.179640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.200107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:59.201769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:59.280276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:59.298047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 138/282] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 19 13:16:59.370904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde49c1-b911-4c14-8cda-e08929a886ea None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.413227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.413803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.414088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.414370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] PATH_INFO: `/v3/users/a6a02ba78ebb4ee9b938021d927aefb7` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.419292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a02ba78ebb4ee9b938021d927aefb7)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.420808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.422579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 145/283] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/users/a6a02ba78ebb4ee9b938021d927aefb7 => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:16:59.428798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.535752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:16:59.559267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ad7f6-070e-4253-aa75-92edb7677161 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.602293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.602947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.603226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.603505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.604236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] RBAC: Adding query filter params (role.id=09b2cc11dfd1423cb451f90ab0447488, user.id=a6a02ba78ebb4ee9b938021d927aefb7, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:59.604582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.606020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.622151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 139/284] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/role_assignments?user.id=a6a02ba78ebb4ee9b938021d927aefb7&scope.system=all&role.id=09b2cc11dfd1423cb451f90ab0447488 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:59.682975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.702700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140716-0b8e-4188-8863-edad1a46e619 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.713330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.714355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.714864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.715343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.733437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:16:59.739788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:16:59.740767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:16:59.741593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:16:59.744133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 146/285] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:16:59.750795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.768512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b1121-66fc-4688-b0f3-6adde20d6400 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.778545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.779191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.779496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.780438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:16:59.780744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.782374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.792000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 140/286] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:59.797414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.812763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2a7d7-db81-49c9-8393-8bcc4edd252d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.822818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.824361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.825011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.825494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.830801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.833121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.835313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 147/287] 10.222.0.39 () {56 vars in 1201 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:59.840963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.856447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57455f15-b396-4981-bd30-0d1291c6b065 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.861299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.861945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.862223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.862498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.863670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.865761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33146d62-4f2c-4700-b151-72019c4ada64 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.871102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 141/288] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:16:59 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:16:59.876739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:16:59.891983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee4ade-369a-4e2a-842b-56cc16c28b51 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:16:59.898155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:16:59.899181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.899672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.900153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae/groups/ba53d755f118436fa61580bb69747fcc/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.903426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c400356aae10410e9cfef1cc92dac4ae, group_id=ba53d755f118436fa61580bb69747fcc, role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:16:59.906352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:16:59.921891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131a33-8829-4b0d-ab7e-0f89722bf3d9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.922197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131a33-8829-4b0d-ab7e-0f89722bf3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.922451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41131a33-8829-4b0d-ab7e-0f89722bf3d9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.924291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 142/289] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:16:59.929651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:16:59.942131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:16:59.942131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:16:59.953332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:16:59.954614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:16:59.968768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 143/290] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:16:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 19 13:16:59.998090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 148/291] 10.222.0.39 () {58 vars in 1463 bytes} [Wed Oct 19 13:16:59 2022] PUT /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae/groups/ba53d755f118436fa61580bb69747fcc/roles/2aaa59501e4146f5adce6e3872912797 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:00.348529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:00.361943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc89444-912d-44c2-ae69-7fc134a40421 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:00.429995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:00.430662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:00.430947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:00.431238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:00.447508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:00.452437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:00.453338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:00.454739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 144/292] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 106 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:00.459924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:00.482106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2379-d736-4da4-852b-427c23718b7b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:00.488709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:00.489634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:00.490120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:00.490655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:00.491529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:00.492033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:00.493847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:00.512575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 149/293] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:00.518009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:00.531803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020dea4-5630-444f-a2e2-f59c9cf10d63 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:00.540109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:00.540756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:00.541035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:00.541304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] PATH_INFO: `/v3/groups/f2c52690b6714f60a99bb3a0c8024777` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:00.547394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] RBAC: Authorizing `identity:get_group(group_id=f2c52690b6714f60a99bb3a0c8024777)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:00.549336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:00.550987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 145/294] 10.222.0.39 () {56 vars in 1201 bytes} [Wed Oct 19 13:17:00 2022] GET /identity/v3/groups/f2c52690b6714f60a99bb3a0c8024777 => generated 229 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:00.556393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:00.567666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b64862-7a79-486e-9c57-f8b56eccb397 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:00.572071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:00.572706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:00.573024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:00.573301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] PATH_INFO: `/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:00.574425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] RBAC: Authorizing `identity:get_project(project_id=cf3ff3a529704cf9a1e0bae3d4c303ad)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:00.576304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9d29e-714c-4996-be2f-8aaec3e7c067 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:00.577935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 150/295] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Oct 19 13:17:00 2022] GET /identity/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad => generated 344 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:00.583781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:00.599815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bc1fd-ea66-4489-a3a7-44de8ecaf98c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:00.604848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:00.605483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:00.605763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:00.606033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] PATH_INFO: `/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad/groups/f2c52690b6714f60a99bb3a0c8024777/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:00.607970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf3ff3a529704cf9a1e0bae3d4c303ad, group_id=f2c52690b6714f60a99bb3a0c8024777, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:00.612819 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e9480-f65d-468a-82ff-d30abc6c14b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:00.692441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 146/296] 10.222.0.39 () {58 vars in 1463 bytes} [Wed Oct 19 13:17:00 2022] PUT /identity/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad/groups/f2c52690b6714f60a99bb3a0c8024777/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:01.242893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c154ce-cae9-48da-847a-fe4738a0f819 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.243214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c154ce-cae9-48da-847a-fe4738a0f819 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.243457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c154ce-cae9-48da-847a-fe4738a0f819 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.244808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 151/297] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:01.250858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479f219-8094-4556-ae3a-f9344b25bdb9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.251146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479f219-8094-4556-ae3a-f9344b25bdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.251427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479f219-8094-4556-ae3a-f9344b25bdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.280059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3479f219-8094-4556-ae3a-f9344b25bdb9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:01.281188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3479f219-8094-4556-ae3a-f9344b25bdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:01.331271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:01.368645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 147/298] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:01.669951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c154ce-cae9-48da-847a-fe4738a0f819 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:01.745883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9cc2-e0de-4c8c-ba5b-6d0bd7f925d5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.746209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9cc2-e0de-4c8c-ba5b-6d0bd7f925d5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.746426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da9cc2-e0de-4c8c-ba5b-6d0bd7f925d5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.748069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 148/299] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:01.764260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.764693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.765038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.766863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:01.802714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:01.804585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:01.820477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c154ce-cae9-48da-847a-fe4738a0f819 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:01.897124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:01.898126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.898644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.898929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.912555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 149/300] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:17:01.919630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-af09e3a2-6762-41b3-8d11-a8d099f27b66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:01.925507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:01.926422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:17:01.927265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:01.929469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 152/301] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:01.935702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:01.964614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfda6-de2d-4db7-9e83-7ca206b42e70 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.965157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfda6-de2d-4db7-9e83-7ca206b42e70 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.965581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfda6-de2d-4db7-9e83-7ca206b42e70 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.967159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 153/302] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:01.972208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:01.972651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:01.973073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:01.999496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:02.000945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:02.123611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.127380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b6a37-911e-4f51-871f-8952fb9e1953 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.130330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 154/303] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:17:02.168299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.169298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.169783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.170257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.171196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:02.171733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:02.173331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:02.185706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 150/304] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:17:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:02.191047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.280993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.306210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.327334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fad53-f832-426e-a1ad-d74d53a3f6c4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.388126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.388754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.389038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.389313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.464482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f0575-d739-46c5-9154-9878ab359688 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.522383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:02.524437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:02.540484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.541967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.542649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.543133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.563265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:02.572531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:02.574056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:02.574949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:02.577558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 151/305] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:02.578599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.607396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.608515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.649429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 155/306] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:02.650448 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.690671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5fe452-bdf7-4a7c-a2a6-d7a6011810fa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.730795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.731899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.732403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.732932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.746095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:02.751773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:02.752787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 13:17:02.753599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:02.756160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 152/307] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:02.757225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.767525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9454b-b7a3-4d3c-a65c-2a081243644f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.802830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.803316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.803594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.803865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.804526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:02.804827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:02.806215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:02.815320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 156/308] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:02.815944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.849066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.871753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:02.897738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275f5d2-5a75-42ae-8a65-473d8a2ab7d8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.931247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e982d3-61d1-4804-8967-82774ce2e2e2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:02.944563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.945792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.946282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.946878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.948761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:02.952263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:02.954523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 153/309] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:02.955541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:02.974344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:02.975030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.975311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.975619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.976296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:02.976649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:02.978018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f1118-708a-490a-8257-a91a31353f4b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:02.986827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 157/310] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:02.987750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981df443-6231-4c12-88e7-f169472bdfd4 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:02.988038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981df443-6231-4c12-88e7-f169472bdfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:02.988359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981df443-6231-4c12-88e7-f169472bdfd4 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:02.989557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 158/311] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:02.990246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981df443-6231-4c12-88e7-f169472bdfd4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.075360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d2104-9e9c-4762-aded-ce6088245c83 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.112052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.121645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.122366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.122682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.122954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.128245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981df443-6231-4c12-88e7-f169472bdfd4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.148881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.167600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.167792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.167901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.168182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.169334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.170839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.172595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 159/312] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:03.173208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.218265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.219762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154c4a3-439d-4ea3-b747-6045d68e5959 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.304251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e995e7c-aecf-4be9-877c-47ff8eca6505 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.347803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 154/313] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:03.349091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.349454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.349764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.354423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.355084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.355367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.355641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.374594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:03.375840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:03.388094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.438144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.449380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.450885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.465732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 155/314] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:03.466766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.569850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 160/315] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:03.571004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.591483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6bdf-0114-45a9-afe0-283cb7beb133 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.635914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.636125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.636391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.636671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.637429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] RBAC: Adding query filter params (group.id=ba53d755f118436fa61580bb69747fcc, role.id=5eaea2af31a1491a9e7ad2736d630ee6, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:03.637740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.639283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.641445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.648326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 156/316] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/role_assignments?group.id=ba53d755f118436fa61580bb69747fcc&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:03.649387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.699583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c497e-6f82-47c6-b83f-59261668dc70 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.742378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.746607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.746607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.746607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.746607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] PATH_INFO: `/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.747347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] RBAC: Authorizing `identity:get_project(project_id=a6026ce3c10a4e3f9da5807a93cd12bd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.748809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.750481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 161/317] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/projects/a6026ce3c10a4e3f9da5807a93cd12bd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:03.756292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.826899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976b0ce-7a45-4e66-9814-af2a7f7460f3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.874709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.875362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.875646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.875923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.877237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.878790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.880907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 157/318] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:03.881801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:03.896824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecfb2e-8e92-43e2-ad04-8479355aa200 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:03.928135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:03.939385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:03.940240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.940393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.940571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.941697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:03.943228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e0af-18a2-4ddd-bef9-8be6fd60b735 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:03.945134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 162/319] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:03.946316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b077-272d-4ce6-a0d2-e77f99f25c8b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:03.948482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b077-272d-4ce6-a0d2-e77f99f25c8b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:03.948845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b077-272d-4ce6-a0d2-e77f99f25c8b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:03.950144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 163/320] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:03.950876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b077-272d-4ce6-a0d2-e77f99f25c8b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.015945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d727d3d-ea77-45ce-abac-90a995fa67e6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.031216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.062386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.063128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.063407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.063674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.078145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:04.079878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b077-272d-4ce6-a0d2-e77f99f25c8b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:04.083824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:04.085175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:04.085986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 158/321] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:04.087183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.126362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.127046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.127327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.127604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.128757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.130188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313f6c3-ec50-4ede-912a-fa4ca99a2f45 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.131866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 164/322] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Oct 19 13:17:03 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:04.132972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.133243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.133520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.152322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:04.153550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:04.211998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d74fc3-3d6b-473d-af42-72d9380b168f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.214388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.226089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 165/323] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:04.226975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.251619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.253153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.253153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.253153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.253642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] RBAC: Adding query filter params (group.id=ba53d755f118436fa61580bb69747fcc, role.id=2aaa59501e4146f5adce6e3872912797, scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:04.253954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.255640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.264504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 159/324] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/role_assignments?group.id=ba53d755f118436fa61580bb69747fcc&scope.project.id=a6026ce3c10a4e3f9da5807a93cd12bd&role.id=2aaa59501e4146f5adce6e3872912797 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:04.264738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.319384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.345140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0493e5e-495d-44b3-b392-eb59994bc79e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.384484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.385118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.385498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.385774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.386466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:04.386999 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.388849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.398062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 166/325] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:04.404982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.412547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c65f3-2778-4e26-8667-c89f506cc55d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.465136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.465760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.466294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.466294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.467050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] RBAC: Adding query filter params (group.id=ba53d755f118436fa61580bb69747fcc, role.id=5eaea2af31a1491a9e7ad2736d630ee6, scope.project.id=c400356aae10410e9cfef1cc92dac4ae) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:04.467363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.469987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.476450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 160/326] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/role_assignments?group.id=ba53d755f118436fa61580bb69747fcc&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=5eaea2af31a1491a9e7ad2736d630ee6 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:04.482084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.532351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.543263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6c9e-193d-466c-b8f3-f0aa5c456364 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.589050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.589676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.589955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.590222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.662783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.667600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3b36a-36c7-420a-b7dd-f11abdf336ff None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.686475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.688249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.711063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.712150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.712636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.713136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.724896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:04.730856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:04.731792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:04.732642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:04.735301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 161/327] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:04.741735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.752883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.811812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 167/328] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:04.818199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.873297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c862766-eb09-4dba-9fa6-677ab10ad783 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.915621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.916512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.917527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.917983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.918882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:04.919381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.920914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.930460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 162/329] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:04.931755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629260c8-82d4-43e9-a40c-0072056be229 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:04.936666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:04.964791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:04.975285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:04.975926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:04.976205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:04.976475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] PATH_INFO: `/v3/projects/c400356aae10410e9cfef1cc92dac4ae` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:04.977673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] RBAC: Authorizing `identity:get_project(project_id=c400356aae10410e9cfef1cc92dac4ae)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:04.979843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:04.981760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 168/330] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/projects/c400356aae10410e9cfef1cc92dac4ae => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:04.987156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:05.017847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:05.062349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c95c5f-3d0d-4c9f-b8de-8377a5a22d69 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.096402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.097008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.097289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.097563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.103977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a0e8-4721-4f73-879e-76425f54c76a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.137818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.138447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.138760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.139041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] PATH_INFO: `/v3/groups/ba53d755f118436fa61580bb69747fcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.140149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] RBAC: Authorizing `identity:get_group(group_id=ba53d755f118436fa61580bb69747fcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.141611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.143267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 169/331] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/groups/ba53d755f118436fa61580bb69747fcc => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:05.149032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:05.167402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.168765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.191706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:05.235101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:05.248099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5f18-ca71-45ae-a4a7-95a3243697a5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.266141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 163/332] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 13:17:05.271911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:05.281936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.282559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.282845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.283115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.283841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] RBAC: Adding query filter params (group.id=ba53d755f118436fa61580bb69747fcc, role.id=2aaa59501e4146f5adce6e3872912797, scope.project.id=c400356aae10410e9cfef1cc92dac4ae) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:05.284144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.285566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.291156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 170/333] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Oct 19 13:17:05 2022] GET /identity/v3/role_assignments?group.id=ba53d755f118436fa61580bb69747fcc&scope.project.id=c400356aae10410e9cfef1cc92dac4ae&role.id=2aaa59501e4146f5adce6e3872912797 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:05.374200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8c7e-3e80-40cc-85e0-b958935762fa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.397599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:05.409114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.409762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.410045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.410319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] PATH_INFO: `/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.411525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] RBAC: Authorizing `identity:get_project(project_id=cf3ff3a529704cf9a1e0bae3d4c303ad)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.413017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.414786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 164/334] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:17:05 2022] GET /identity/v3/projects/cf3ff3a529704cf9a1e0bae3d4c303ad => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:05.421065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:05.525473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca60391-96b1-4a4b-9bf6-2270f551b982 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.559833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.560473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.560752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.561032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] PATH_INFO: `/v3/groups/f2c52690b6714f60a99bb3a0c8024777` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.562132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] RBAC: Authorizing `identity:get_group(group_id=f2c52690b6714f60a99bb3a0c8024777)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.563608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47a36e-9856-4c0e-823e-32b35b0092cb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.565184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 171/335] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Oct 19 13:17:05 2022] GET /identity/v3/groups/f2c52690b6714f60a99bb3a0c8024777 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:05.571143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:05.672465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae560a-0732-40fb-9828-e5ebc5c7cd0d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:05.703869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:05.704461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:05.704738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:05.705006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:05.705715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] RBAC: Adding query filter params (group.id=f2c52690b6714f60a99bb3a0c8024777, role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=cf3ff3a529704cf9a1e0bae3d4c303ad) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:05.706017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:05.707466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b794-2023-4894-9c51-b3929d19ebdb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:05.713238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 165/336] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Oct 19 13:17:05 2022] GET /identity/v3/role_assignments?group.id=f2c52690b6714f60a99bb3a0c8024777&scope.project.id=cf3ff3a529704cf9a1e0bae3d4c303ad&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:08.334133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc15ae-0e86-496d-bd25-03e84b246683 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.334980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc15ae-0e86-496d-bd25-03e84b246683 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.334980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc15ae-0e86-496d-bd25-03e84b246683 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.336599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 172/337] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:08.342227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32597f-64ca-46ed-9d36-5e3f345ee4a2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.343092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32597f-64ca-46ed-9d36-5e3f345ee4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.343569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32597f-64ca-46ed-9d36-5e3f345ee4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.371490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5f32597f-64ca-46ed-9d36-5e3f345ee4a2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:08.373037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5f32597f-64ca-46ed-9d36-5e3f345ee4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:08.438366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:08.457734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 166/338] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:08.733832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f3313-7feb-427a-83be-6ce69aad9e45 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.734017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f3313-7feb-427a-83be-6ce69aad9e45 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.734309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f3313-7feb-427a-83be-6ce69aad9e45 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.734922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af4f5-aa02-40cc-b848-c46fe1e41ac5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.735212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af4f5-aa02-40cc-b848-c46fe1e41ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.735488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af4f5-aa02-40cc-b848-c46fe1e41ac5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.735936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 173/339] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:08.736802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 167/340] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:08.741864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.742154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.742435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.745523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:08.745812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:08.746090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:08.777998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:08.779234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:08.781954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:08.783143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:08.806683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:08.867059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 174/341] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:08.867996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:08.924236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 168/342] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:08.991050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcc014-fef6-46cd-8a84-fc80f2a783b5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.035074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.035960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.036419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.036834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.047843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.049541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-1bd17120-6227-45dc-83f8-3a902a51a3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:09.065151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:09.066353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:09.067531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:09.067531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.067531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:09.068902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 175/343] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:09.079820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.142920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1e24-03c6-475c-88b8-2c2df9cd7366 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.143433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1e24-03c6-475c-88b8-2c2df9cd7366 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.143852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1e24-03c6-475c-88b8-2c2df9cd7366 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.145519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 176/344] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:09.150348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.150676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.151024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.161072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.179584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:09.184640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:09.230890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bbee9-8182-4566-a312-dcb691f9c118 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.246680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.271559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 177/345] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:09.272140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.274785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.274909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.274909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.274909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.274909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:09.274909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.277800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.295221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 169/346] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:09.302455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.403673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffe33-0aae-4d42-bc2c-c32d99abb29e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.435020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.440010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.440652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.440931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.441203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.445200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c21e9a-47d0-4fe4-ba95-b622a853ec3c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.452101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.453552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:09.471312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:09.472451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:09.473367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:09.473367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.473367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:09.473367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 178/347] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:09.473367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.493633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.494489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.495514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.495945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.497369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.499671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.575918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 170/348] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:09 2022] POST /identity/v3/users => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:09.576756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.582548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062e910-44f8-4465-a757-8655ebfd175b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.625570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.626302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.626669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.626943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.627763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.641275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.643022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:09.662310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:09.663340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:09.664264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:09.664264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.664264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:09.669281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 179/349] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:09.670582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.711382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.732634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e855d9-3817-432c-b849-5c22a6c1c205 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.772226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.772757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.773042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.773665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.773944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:09.774242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.775636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.786842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4030c-5493-4259-856a-27d076c0a0dc None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.790320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 171/350] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:09.790816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.825480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.826231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.826544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.826974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.840000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.841457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-174445f8-917b-456f-860f-34eca69f95e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:09.856178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:09.857110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:09.857962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:09.857962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:09.857962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:09.857962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 180/351] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:09.858832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351b592-81d4-442a-815c-8c45c593a000 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.858832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351b592-81d4-442a-815c-8c45c593a000 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.859012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351b592-81d4-442a-815c-8c45c593a000 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.860390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 181/352] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:09.863740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351b592-81d4-442a-815c-8c45c593a000 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.900698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.911625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28199ce6-fc2c-4263-b749-a8dfb99b8964 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:09.948579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:09.949189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:09.949481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:09.949736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:09.950372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:09.950726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:09.952078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:09.961213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:09.965942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 172/353] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:09.967369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:09.976190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351b592-81d4-442a-815c-8c45c593a000 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.013775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.014401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.014713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.014989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.015643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:10.017140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb1f14-e730-48da-993a-3760766dddf8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:10.082835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e56c3c-2c6f-4fb6-a4b5-5722b8415d14 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.117181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.117875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.118158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.118429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.119105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:10.119406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:10.120798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:10.139046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 182/354] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:09 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:10.139046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.139046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.139046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.139469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 173/355] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:10.139469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:10.158053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:10.160235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:10.161524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:10.258156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c71f13-7601-4a5e-8d63-0e99eb369f5d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.258930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 183/356] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:10.259774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:10.349535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.350493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.351046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.351530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.352184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:10.353636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:10.359274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:10.392028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8252d8-344c-4e39-9055-18cede1c7778 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.428415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.429173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.429451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.429726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.430371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:10.432151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:10.464933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 174/357] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:10 2022] POST /identity/v3/users => generated 248 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:10.569955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 184/358] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:10 2022] POST /identity/v3/users => generated 249 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:10.583454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:10.677093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:10.721563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012550e-03b6-4971-87ce-8c3ad25845dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.759980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.760707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.760984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.761258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.761954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:10.763376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:10.806569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 13:17:10.806569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 19 13:17:10.806569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [parameters: {'id': '527e251a34d84f9b9adad8655389d3f7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 19 13:17:10.806569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:17:10.814252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return ref.to_dict() Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:17:10.815220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 13:17:10.816137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 13:17:10.816966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:17:10.817907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:17:10.820536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '527e251a34d84f9b9adad8655389d3f7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:10.822491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 19 13:17:10.826714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:10.828854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 175/359] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Oct 19 13:17:10 2022] POST /identity/v3/roles => generated 144 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:10.828990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:10.931953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c861f-059b-4452-a4e0-b1ab565c78c3 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:10.962903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:10.968277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:10.968923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:10.969198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:10.969467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:10.981556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-0cda7859-c045-4794-9cf5-30394a133b33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:10.986567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:10.987462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:10.988275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 185/360] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:10.993354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:11.102476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77e960-f7ef-4f60-ac25-0ed99e8e3269 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:11.147686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:11.160014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:11.160692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:11.166094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:11.166094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:11.166094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:11.166094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:11.166094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1842155-25b9-4f93-91c8-4c0b78f1aebe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:11.173073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 176/361] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:11.881462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ea76-8c3a-42ee-82fb-dd51b1882167 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:11.881972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ea76-8c3a-42ee-82fb-dd51b1882167 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:11.882047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ea76-8c3a-42ee-82fb-dd51b1882167 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:11.883369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 186/362] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:11.894268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:11.894717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:11.894893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:11.927785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:11.929027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:12.021138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 177/363] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:12.311496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ea76-8c3a-42ee-82fb-dd51b1882167 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:12.445591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ea76-8c3a-42ee-82fb-dd51b1882167 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:12.490725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:12.491741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:12.492216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:12.492714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:12.511356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:12.517021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:12.517900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:12.518729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:12.520875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 187/364] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:12.527311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:12.659282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255b985-ddf0-458b-8abc-1751f933df45 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:12.688905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:12.701660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:12.714683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:12.715176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 178/365] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:12.719508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:12.771801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:12.831261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4626a2-6100-4b10-90dc-2dfccb7c4413 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:12.882232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:12.882914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:12.883197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:12.883472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:12.899978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:12.902022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f542adb4-eb6c-40a9-a1e1-3b29e1f578ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:12.924912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:12.925825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:12.927303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:12.927303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:12.927303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:12.927303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 188/366] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:12.931768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:12.981935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326def80-625e-4091-b42c-3b2e4287e611 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:12.981935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326def80-625e-4091-b42c-3b2e4287e611 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:12.981935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326def80-625e-4091-b42c-3b2e4287e611 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:12.981935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 189/367] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:13.007687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d608a-8057-4a46-beab-0bb5c93e2d1e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.007976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d608a-8057-4a46-beab-0bb5c93e2d1e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.008255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d608a-8057-4a46-beab-0bb5c93e2d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.054678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1b7d608a-8057-4a46-beab-0bb5c93e2d1e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:13.055905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1b7d608a-8057-4a46-beab-0bb5c93e2d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:13.100580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe422af-545e-4a40-b8ff-3933d6c91805 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.137353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.138013 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.138299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.138597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.139251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:13.139554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:13.140925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:13.153741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 179/368] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:13.159326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.190395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 190/369] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:13.192173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74534ad8-b80d-4a5b-9ab2-e9fcfb7e1bb5 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.192653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74534ad8-b80d-4a5b-9ab2-e9fcfb7e1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.193228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74534ad8-b80d-4a5b-9ab2-e9fcfb7e1bb5 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.199427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 191/370] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:13.201100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.201603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.202163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.219737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.223210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:13.224597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:13.253138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.278729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f60eec-acac-469c-9247-8cf1abc1f3d4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.300831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 192/371] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:17:13.315205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.315867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.316155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.316429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.325998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:13.327513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-26766ce9-0019-4941-8a4c-b7fc11d46512 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:13.342034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:13.343096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:13.344009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 180/372] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:13.351076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.471566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569546d9-39a5-4eea-9838-347e940dd482 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.471753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569546d9-39a5-4eea-9838-347e940dd482 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.472035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569546d9-39a5-4eea-9838-347e940dd482 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.473366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 181/373] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:13.480164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.480449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.480722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.511497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbfcb9-3acb-4d96-bbba-70d01b9b1f57 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.511679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:13.512714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:13.553810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.554432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.554735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.555009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.555743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:13.556048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:13.558671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:13.605187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 193/374] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:13.605678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 182/375] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:13.606325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.614630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.654970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.675198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.752493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fad86-8d15-45fd-a6dc-b16374beded4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.759492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa002eca-e7a8-42ad-874c-5cbdd5cbb9bb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.795447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.796250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.796604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.796707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.796963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.797307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.797762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.798047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.807656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:13.809196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:13.810477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:13.815946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:13.816834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:13.818248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 183/376] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:13.823168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:13.830189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:13.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:13.832600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:13.832600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:13.832600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:13.832600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 194/377] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:13.832600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.907325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.937253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616fc13-9475-406d-88c7-30ef1cba4d67 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.938060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:13.944752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e8178-0e0b-4b4e-954a-1df2ce1dd232 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:13.974940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.975673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.975955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.976343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.977176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:13.977424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:13.978896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:13.982749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:13.983387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:13.983664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:13.983932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:13.992077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 195/378] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:13.992077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:13.998265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:14.003951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.004842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:14.005686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 184/379] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.010899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.109664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8073de-c3eb-417b-ae2e-08eec38fc393 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.124988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8610-4d83-4d5f-9cd2-2e0d7d030fa5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.146593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.147209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.147488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.147757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.148414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.148699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.150079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.152357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.162894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.168029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.168029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.168029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.172706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 196/380] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.172706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.178411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:14.183346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.184232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 185/381] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.185804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.202365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.270018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6178b9-be9d-414c-b09d-1592621dc25a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.285532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cafa8-d514-4599-bddd-9f3af8a3c3d9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.302844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.303468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.303745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.304015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.314052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.315508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.318083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.318705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.318987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.319260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.319917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.320220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.321576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994612c3-9037-4c17-9f68-b05e3dd54834 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.329655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 186/382] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:14.334612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.335453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:14.336289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:14.336289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.336289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:14.336536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd413-71db-401b-9e91-bead4d0441a4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.336536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd413-71db-401b-9e91-bead4d0441a4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.336658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 197/383] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.336737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd413-71db-401b-9e91-bead4d0441a4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.336805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.336880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 187/384] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:14.336880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd413-71db-401b-9e91-bead4d0441a4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.380335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.412695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.431811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131d639-2d51-4f37-8eb9-fbd21d2e8206 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.436110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd413-71db-401b-9e91-bead4d0441a4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.465140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.465757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.466035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.466333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.470672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.471269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.471546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.471815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.472473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.472772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.472984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.474136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.474407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.481990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 188/385] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.483370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-99172c46-ad22-4616-a5d1-98b0d1fad59c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:14.486178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.487063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:14.488026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 198/386] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.488950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.489124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.489350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.508560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:14.509736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:14.574382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 199/387] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:14.575223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.583554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837668a-9ad6-4fd9-a906-95848418786a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.614262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.618929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.620464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.620750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.621021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.630744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.634493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.637212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:14.652691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.653613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:14.655584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:14.655584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.655584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:14.655584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 189/388] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.655584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.684594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc870f-6783-45ca-bb98-3534a23c5219 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.720749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.721345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.721626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.721898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.722578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.722881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.724271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.735381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 200/389] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.736165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.767255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76ef00-11e5-447a-81ea-e0fae4197b0b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.803180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.803800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.804077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.804512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.815865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.817292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:14.832440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:14.833495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:14.834949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:14.834949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:14.834949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:14.834949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 190/390] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:14.834949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.842328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.848305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32e9f9-178f-4225-b693-68e667a5d89e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.878343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:14.882614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.883223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.883502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.883773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.884467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.884774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.886226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.903700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 201/391] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.904598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:14.934599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356894-4655-437c-ab84-aa8aa6b33170 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:14.967238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:14.967860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:14.968137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:14.968414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:14.969051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:14.969358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:14.970810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:14.982866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 191/392] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:14.983697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.007586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa8f8-f96e-48b8-b8a2-a4f5fd65ff49 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.042334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.046432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.046432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.046432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.050425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.051933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:15.065292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:15.066219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 202/393] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:15.067190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.080692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd5f1-5c42-4e61-b0cc-0f4adbf1f069 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.082315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.089013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.115536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.116190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.116471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.116747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.117385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:15.117688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.119098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.130680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 192/394] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:15.131450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.173908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdc63b-ed7a-46f4-8c17-5182fced1309 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.208501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.209136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.210001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.210287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.221463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.222944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.230859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244e4d5-bfeb-4f9b-a281-f74d936ade06 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:15.237692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:15.238870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:15.239731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:15.239731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:15.239731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:15.239731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 203/395] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:15.239946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.266772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.267427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.267708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.267980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.282400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=f025b86069754817868da5d0b19e3de0, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.283894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.304785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.337705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49628007-bc27-4dc1-98ac-b36a66991404 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.362972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 193/396] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:15 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:15.363749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.378411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.379122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.379660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.379960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.380303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.391147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.393195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:15.408516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:15.409491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:15.410417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 204/397] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:15.411028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.477314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf4d43-4942-49da-bd0e-b53961fdac9e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.514645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.515294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.515579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.515854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.516594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:15.516901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.518349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.519034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0cf0a-dc35-4170-98a9-0272dff2d8a1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.548727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.557702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.558476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 194/398] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:15.558627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.558703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.558771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.559150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.570188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.571787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:15.583510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:15.584697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 13:17:15.585612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:15.590036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 205/399] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:15.590897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.640234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.672529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638823-d577-4f0b-8c82-f06b866af646 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.697121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087805-73d9-4694-811f-7c8da1f8eec1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.709856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.710635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.710981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.711321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.712039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:15.712341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.714278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.730563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 195/400] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:15.730563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.736848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.737597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.737976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.738346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.739213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:15.739621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.741177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.764631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 206/401] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:15.765304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.822316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.832399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911482a9-b750-4828-b42b-6e01c7b70e12 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.864338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:15.872576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.873253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.873517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.873941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.880589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467391a9-b9fb-450d-b8e1-f5a8e6a54237 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:15.884900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.886982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:15.904614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:15.905706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:15.906587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:15.906587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:15.906587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:15.906587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 196/402] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:15.906587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:15.926233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:15.926888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:15.927167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:15.927441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:15.928177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:15.928483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:15.929905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:15.947335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 207/403] 10.222.0.39 () {56 vars in 1139 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:15.948935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.024272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92808-807a-4297-8930-13c122321ce3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.056563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d8b9-f2a4-4f3f-9fe1-ed6053d94bf5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.059587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.061458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.062063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.062371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.062680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.063317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.064796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.095153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.095793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.096070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.096337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.106640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.108133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:16.122802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:16.123912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:16.124944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:16.124944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:16.124944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:16.124944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 208/404] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:16.126281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.144047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 197/405] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:15 2022] POST /identity/v3/users => generated 247 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:16.145084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.192064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.277078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087ab6-750d-469d-807a-be3bae152116 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.291442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302296bf-7c10-46ca-8e10-913e7358c9a3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.317335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.318024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.318422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.318620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.319279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:16.319586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.321001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.335724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 209/406] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:16.336540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.344970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.346188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.346669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.347110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.361499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.366427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.367716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:16.386363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:16.387477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:16.388342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:16.388342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:16.388342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:16.388342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 198/407] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:16.389197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.446913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd77718-1373-400e-aafd-9961e38e3314 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.499888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.502041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.502939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.503219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.503494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.504305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:16.504903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.506567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.511462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91962-215f-4536-983e-b3ec826a1f7d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.522438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 210/408] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:16.523446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.558467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.559281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.559631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.559973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.568955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.571460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:16.597726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:16.598652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.599509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:16.599938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 199/409] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:16.601142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.614985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.638934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a87a-1d7c-4f2e-92e7-b2f1f5195f80 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.675064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.675896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.676238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.676579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.677325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:16.677697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.679195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.692748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 211/410] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:16.693717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.744014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21e8a-1fe7-4755-b9fd-a23aba9d5164 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.783456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.784454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.785331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.786051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.795945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.798223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.807599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5d7ed-0a83-4c89-86cf-9996dc0572f6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:16.813889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:16.814787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:16.815646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 200/411] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:16.816600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.829259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.860596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.861842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.862621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.863102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.864168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:16.865125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.867335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.870537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:16.885484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 212/412] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:16.886902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:16.924810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1317-e65f-4f74-82d0-b15fb01882ff None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:16.961116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:16.961554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:16.961831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:16.962101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:16.970078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:16.971401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:16.983948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:16.984835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:16.985688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 201/413] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:16.986621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:17.037305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe291c60-f2e1-4fb5-aee5-75ad7f47617e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:17.069226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:17.087129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:17.089099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.089531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.089952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.090857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:17.091309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:17.093632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:17.104614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae78c8-dbb5-4f66-946f-de641999c094 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:17.110607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 213/414] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:17.111722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:17.142584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:17.143416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.143791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.144173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/39feed1e4a48413d823bf0ece9b8f59c/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.161220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=39feed1e4a48413d823bf0ece9b8f59c, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:17.162877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:17.176591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:17.219734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ffdf3-1204-4c55-a5a2-7515b9079546 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:17.249097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 202/415] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:16 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/39feed1e4a48413d823bf0ece9b8f59c/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:17.250714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:17.272514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:17.273156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.274149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.274420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.275209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:17.275520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:17.278373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:17.300251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 214/416] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:17.305145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:17.418573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848efe54-e440-4241-8691-fc1d7e1cac37 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:17.444090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:17.463097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:17.464332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.464631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.464886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/bf272634ade542739c5cfe3139bb201a/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.485316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=bf272634ade542739c5cfe3139bb201a, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:17.487217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389d512-dd0e-4aba-96a0-44e22e8d983f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:17.527926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426950-f15a-4bb5-9c29-3a5d3a7ef4fb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:17.545381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:17.555302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 203/417] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:17 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/bf272634ade542739c5cfe3139bb201a/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:17.559590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8656e-bcde-4000-bd9d-acf9fd74b1f6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.559590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8656e-bcde-4000-bd9d-acf9fd74b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.559590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8656e-bcde-4000-bd9d-acf9fd74b1f6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.565884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 204/418] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:17.566066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:17.566787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.567119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.567401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/0f2a170d431b4524b18178757fa3d627/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.577036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:17.577358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:17.582211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:17.586982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=0f2a170d431b4524b18178757fa3d627, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:17.588615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:17.607572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:17.608865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:17.681333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 215/419] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:17 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/0f2a170d431b4524b18178757fa3d627/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:17.693217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 205/420] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 19 13:17:17.973519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:18.082891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d712928-98c8-45b5-b0bc-34069cf30988 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:18.119106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:18.120457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.120648 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.120925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.131853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:18.136933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:18.137821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:18.138679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:18.142619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 216/421] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:18.144265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:18.249873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:18.253392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57231d2e-5cd1-4a97-a46c-b102b91e5ce9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:18.289598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:18.290227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.290526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.290807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.292030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:18.292335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:18.293729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:18.302489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 206/422] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:18.308368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:18.340579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:18.420509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8321dba-ff8b-4e1d-8f1a-91c45f3c2e8a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:18.460590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:18.461226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.461509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.461780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.472582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:18.474028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-aa3da40e-a8ee-46bd-986e-f1ce07fe367d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:18.489126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:18.490072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:18.490963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:18.490963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:18.490963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:18.490963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 217/423] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:18.495959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:18.680498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e0a7-fe7b-4864-8ba4-e1eeaaa6a511 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.680993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e0a7-fe7b-4864-8ba4-e1eeaaa6a511 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.681379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e0a7-fe7b-4864-8ba4-e1eeaaa6a511 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.683128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 218/424] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:18.688846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.688846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.688846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.707750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:18.708960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:18.745161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b12edb-c323-451e-a31b-fa1f03857582 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:18.798200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 219/425] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:17:18.825347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:18.825968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:18.826243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:18.826533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:18.827185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:18.827484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:18.828852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:18.848889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 207/426] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:18.855217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:18.868838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.001248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693f348-f0bf-4268-a9fe-ecb88581ee9d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.052773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.053412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.053750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.054035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:19.064047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:19.065624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:19.080002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:19.085715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:19.086671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 220/427] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:17:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:19.087238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.127623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.161455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.259503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb23b34-3563-4002-887b-ec5d46a4a285 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.274729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.325148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.325763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.326045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.326321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:19.327081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:19.327387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:19.330915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:19.340592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9b-a23d-46cc-8be9-d670f1f95078 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.366677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 208/428] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:19.378538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.423247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.423896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.424178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.424455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:19.468278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:19.472600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:19.473524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:19.474460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:19.475312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:19.478982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 221/429] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:19.492146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.495172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49875a70-253e-4079-a1a0-dfe888515ee9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.552770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784778e-7030-4a8c-919b-9848209427df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.553393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784778e-7030-4a8c-919b-9848209427df None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.553676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784778e-7030-4a8c-919b-9848209427df None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.553948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784778e-7030-4a8c-919b-9848209427df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:19.673753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561606-2bb1-4284-a7dd-c4bf61ecd5cf None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.682754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784778e-7030-4a8c-919b-9848209427df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:19.684246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784778e-7030-4a8c-919b-9848209427df None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:19.726782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.736298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.736917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.737201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.737477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:19.738385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:19.738714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:19.742148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:19.763887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 222/430] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:19.768425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.827062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:19.868312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 209/431] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:19.876183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784778e-7030-4a8c-919b-9848209427df None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:19.948842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b5b9-fa93-45be-82aa-b920c386c20f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:19.993627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:19.994916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:19.995206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:19.995481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.006241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.007971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.016276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784778e-7030-4a8c-919b-9848209427df None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-3482fafc-d637-4f73-a130-19fa08dd936c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:20.028199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:20.029285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:20.030662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:20.030662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:20.030662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:20.030662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 223/432] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:20.030950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460c0a-9fd6-4364-9ba4-d219d1644001 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.031033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460c0a-9fd6-4364-9ba4-d219d1644001 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.031252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460c0a-9fd6-4364-9ba4-d219d1644001 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.036905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 224/433] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:20.037875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d460c0a-9fd6-4364-9ba4-d219d1644001 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.060229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.061100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.061591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.062029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.077575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.080385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.088940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4ca0e25a-c49e-4624-b716-6bd371ab2a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:20.100488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:20.101697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:20.102759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:20.102759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:20.102759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:20.109758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 210/434] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:20.109758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.109758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.109758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.132061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:20.133283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:20.160522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.173269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d460c0a-9fd6-4364-9ba4-d219d1644001 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.218680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 211/435] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:20.219791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.224805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.225808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.226298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.227870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.228895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:20.229532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.231286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.246864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 225/436] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:20.252378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.366258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaaab6-fef8-4ea8-8c3c-07f930760ec8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.372336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467bbc-daca-43b0-ad07-00077cb4f6cd None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.374003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.400766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.401524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.402148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.402431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.403116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:20.403422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.404790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee979e9-34c4-422f-a6d6-961df6ca8be6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.410567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.411492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.411944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.412365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.415815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 212/437] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:20.417535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467ce65-9040-4839-ae40-bc75853fb132 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.417821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467ce65-9040-4839-ae40-bc75853fb132 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.418100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467ce65-9040-4839-ae40-bc75853fb132 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.423579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 213/438] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:20.423579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2917f-d04a-4a9c-887b-c7834f9920e7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.423579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2917f-d04a-4a9c-887b-c7834f9920e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.423579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2917f-d04a-4a9c-887b-c7834f9920e7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.423579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 214/439] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:20.424117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2917f-d04a-4a9c-887b-c7834f9920e7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.425357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.427121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-d4c3849d-9d8c-4505-89db-0c0447a7dc66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:20.441919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:20.442874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:20.443795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 226/440] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:20.445000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe30dd-4821-4ba7-b18e-8c919e36f6c8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.445000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe30dd-4821-4ba7-b18e-8c919e36f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.445000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe30dd-4821-4ba7-b18e-8c919e36f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.456156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.462749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5bfe30dd-4821-4ba7-b18e-8c919e36f6c8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:20.464559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5bfe30dd-4821-4ba7-b18e-8c919e36f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:20.534957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 227/441] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:20.536290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.536583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.536878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.542088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2917f-d04a-4a9c-887b-c7834f9920e7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.557323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:20.558518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:20.583072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.583697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.583979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.584504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.591827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.593266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:20.604982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:20.605872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:20.606767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 215/442] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:20.607226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.634630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 228/443] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:20.635777 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.674836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.705264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.726499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147617-4e5c-4d1f-a428-6c7ba64ccc04 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.761761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.762603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.762952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.763301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.764207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:20.764583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.766202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.774069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d3fc9-bac4-4d43-8e31-e1beebc5236b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.789594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 216/444] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:20.790422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.820134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.820775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.821059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.821332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.831952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:20.836994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:20.837887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:20.839270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 229/445] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:20.840503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.908027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8661f-045b-4405-80a9-0f246411d19b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.924662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.942456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfb990-0a1b-4232-a81d-eb122e5e290f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:20.944464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.945080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.945365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.945638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.946328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:20.946657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.948678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:20.964196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:20.967670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 217/446] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:20.968518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:20.982238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:20.982888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:20.983166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:20.983442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:20.993151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:20.994646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:21.011336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.012241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:21.013104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:21.013104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.013104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:21.013104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 230/447] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.013318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.073409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270e45e-8af6-4696-a9ba-0b8b4e3ae70c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.107901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.108526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.108811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.109088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.113409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091489fc-594a-410e-91c5-25f0c8e88e0e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.119208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:21.124142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.125057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:21.125856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 218/448] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.126564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.148727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.149412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.149697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.149966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.151186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.151494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.152848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.154280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.165286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 231/449] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:21.165286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.186139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.225954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9423eb-5ac4-4e1c-9773-a41f59dce20c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.259548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.260710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.260990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.261260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.262396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfc40e-a8ce-492b-b741-e4fbf695dbd7 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.271149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:21.276357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.277249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 219/450] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.278269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.299837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.300441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.300992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.300992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.301689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=f025b86069754817868da5d0b19e3de0) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.301994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.303451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.309571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 232/451] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/role_assignments?user.id=f025b86069754817868da5d0b19e3de0&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:21.310334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.378021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.399621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54777ad0-e15b-427c-9d7c-937e7c46a084 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.423743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.423998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699e7-6709-4168-8ed8-2978ac9a1adb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.440120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.440776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.441117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.441402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.442221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.442559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.444152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.464423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 220/452] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:21.464423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.465762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.466398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.466712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.466986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.467672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.467975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.469411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.478186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 233/453] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:21.479096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.581227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457599ed-539e-4fca-b5b9-787e7c422586 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.586215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b53b3-2c65-40a8-99f9-3797871add60 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.610499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.618619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.619309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.620085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.620363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.623160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.623772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.624055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.624333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.633145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.633145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.633145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.633589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.633589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.635810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 234/454] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:21.636757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:21.648903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.649842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:21.651146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:21.651146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.651146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:21.651146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 221/455] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.652863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.687637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.748036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b666f-021d-4293-9133-8b8bf342985f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.764147 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4741ac-36a0-4c11-aa00-1d9c0a719e2e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.784256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.784895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.785184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.785462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.794023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.795521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.800715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.801503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.801786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.802060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:21.806840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.808198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:21.809054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 235/456] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.809515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.816895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.818931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:21.838413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:21.839379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:21.840239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:21.840239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:21.840239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:21.842822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 222/457] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:21.843912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:21.856794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.922456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b259b7-d0ad-4a87-a45b-3091521418a5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.932620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:21.949572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712d73f-50a8-488f-a17f-a655e5161e57 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:21.967891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.968607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.968889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.969161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.983099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.984573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:21.986048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:21.986957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:21.987680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:21.987966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:21.988610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:21.988917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:21.990301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:22.001444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:22.002401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:22.004165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:22.004165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:22.004165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:22.004165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 236/458] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:22.004165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.014097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 223/459] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.014097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.114129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.119843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1126bd8-d730-4ee2-9ab9-d2e1663a63b4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.157070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.157673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.157953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.158223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.158901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb345-23f4-4189-9244-a0bb5f4150ec None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.159012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:22.159235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.160669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.178098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 237/460] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.178098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.202794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.203340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.203616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.203895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.204565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:22.204874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.206296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.220240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 224/461] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.220966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.226012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.288264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75564-3a44-4c32-b76e-888ecf3dee79 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.324923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.325531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.325818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.326087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.326760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:22.327065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.328431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.329526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c338f3-4228-465a-b2f2-953f549bb9c9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.344916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 238/462] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.344916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.363701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.371052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.376547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.376547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.376547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.379854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.381292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:22.393134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:22.394053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 19 13:17:22.395009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:22.398954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 225/463] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:22.399774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.453421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81ac89-2a1b-4e1b-96a1-bb4fb3e5d93c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.466153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.490152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.490888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.491169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.491454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/7498ce3216d1411c961279f321418b7b/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.506447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=7498ce3216d1411c961279f321418b7b, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.506680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a266-44b7-4aa6-a086-16b85ca58ae2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.508093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.541385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.542009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.542292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.542594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.550116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.551612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:22.563350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:22.568003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 226/464] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:22.568996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.600129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 239/465] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:22 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/7498ce3216d1411c961279f321418b7b/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:22.601171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.687660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a5f3e-e6a5-403e-ba2a-a93c118e0b4e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.700973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.716695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:22.729482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.730465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.730931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.731380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.740658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.742467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.748697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247355-2f42-4a55-b68b-01db6bcd18be None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:22.756633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:22.757785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 13:17:22.758946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:22.761737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 227/466] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:22.763192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.808215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.808984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.809362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.809730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.811122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:22.811599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.813620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.842664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 240/467] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.843793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.890468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c16c-8169-4450-9305-9cac3b86f208 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:22.929971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:22.930645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:22.930933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:22.931210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:22.932485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:22.932796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:22.934300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:22.953198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 228/468] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:22.954092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:22.981679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f7234-d74e-47de-8547-910f8e9757fc None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.006481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.014489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.028295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.029636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.030237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.030827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.031890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:23.032371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.034463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.058210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 241/469] 10.222.0.39 () {56 vars in 1139 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:23.058210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.082178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d532cea-9e37-43fb-835f-f3c106960d23 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.121300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.122058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.122412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.122805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.215051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecd3cf-01c6-4508-a7d4-ee60f778cb4c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.227404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.229402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.250940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.251615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.251981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.252396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.284600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.305374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.326178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.327988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.364387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 229/470] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:23.364387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.468336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 242/471] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:23.469532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.481237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9f8b6-ed45-4457-98f1-67651ddef245 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.520638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.521308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.521591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.521868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.534554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.562127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.580254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd30051-8763-445a-92be-ee7fcf4e3556 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.594012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.595540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3507c-5886-4669-8074-d01b6722ebe9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.615784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.616415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.616691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.616960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.627617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.629046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:23.644142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:23.645049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:23.645938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:23.645938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:23.645938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:23.645938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 243/472] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:23.646497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 230/473] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf5f9-ce5d-47ee-856a-ebdcdbb27559 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf5f9-ce5d-47ee-856a-ebdcdbb27559 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf5f9-ce5d-47ee-856a-ebdcdbb27559 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 231/474] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:23.711323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbf5f9-ce5d-47ee-856a-ebdcdbb27559 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.757353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.763935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c940a3-fb18-45a2-9561-28563839d83c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.802249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.802919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.803205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.803488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.805524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:23.815888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.819868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.819972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbf5f9-ce5d-47ee-856a-ebdcdbb27559 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:23.832277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:23.833406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:23.834759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:23.834759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:23.834759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:23.835238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 244/475] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:23.836043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.863938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.864701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.864982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.865256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.866228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:23.866557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:23.867928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f60429-f6cb-4550-a3d6-f39729487d0f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:23.884179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 232/476] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:23.884179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.884179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.884179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:23.899782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:23.900989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:23.949022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316551e-e0c8-44e1-850d-301a54d1df25 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:23.972672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 233/477] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:23.973798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:23.990415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:23.991254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:23.991605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:23.991948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.004173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.005861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.016836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:24.022480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:24.023662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:24.024737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:24.024737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:24.024737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:24.029766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 245/478] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:24.030663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.082487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.098051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8795-76d1-4056-9883-82ffbda276b5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.135811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.136592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.137029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.137353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.138142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.138604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.140189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.152613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 234/479] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.153736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.156750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1163e-eec4-49b2-8dc4-7bb25dcf0ed9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.199994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.200891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.201620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.202176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.210808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.213656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:24.228194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:24.229787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.230865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:24.233707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 246/480] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:24.234756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.262836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.274856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f212a5-143a-40f5-9731-e3d2b7204b1d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.310543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.311148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.311426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.311695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.312327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.312624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.313992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.325894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 235/481] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.326661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.346392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.350415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdede2-0eea-4028-bcdd-9804b1aa783b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.384576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496ab9-5856-4332-a066-368069236d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.385204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496ab9-5856-4332-a066-368069236d5f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.385482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496ab9-5856-4332-a066-368069236d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.385752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496ab9-5856-4332-a066-368069236d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.392607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91496ab9-5856-4332-a066-368069236d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.394039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91496ab9-5856-4332-a066-368069236d5f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-91496ab9-5856-4332-a066-368069236d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:24.405771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:24.406685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:24.407560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 247/482] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:24.408030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496ab9-5856-4332-a066-368069236d5f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.434532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903145ec-aec7-45b9-aa48-33be8e6b946d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.472740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.473343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.473727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.473894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.474603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.474909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.476321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.489004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 236/483] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.489739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.504370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.519668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496ab9-5856-4332-a066-368069236d5f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.555406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.556154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.556431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.556738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.566906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:24.572712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:24.573810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 248/484] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:24.575799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.597221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.604892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ce5-da72-4c35-bc7a-5a410992f3ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.640206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.640997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.641387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.641764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.648362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.649798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:24.661343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:24.662393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 237/485] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:24.663496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.689674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa95fd4-5608-4704-a319-f07997319983 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.729547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.730436 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.730890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.731343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.732364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.732678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.734115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.740457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.747975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 249/486] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.748807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.777127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39981eb-e388-4d5b-872e-fc4518a3b656 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.813993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.814640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.814927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.815205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.815971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=39feed1e4a48413d823bf0ece9b8f59c) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.816280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.817756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.838825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:24.839031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 238/487] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/role_assignments?user.id=39feed1e4a48413d823bf0ece9b8f59c&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.840663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:24.872829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913fc5-ab30-4f4c-acf6-86ec181fb222 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:24.918742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:24.919654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:24.920011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:24.920391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:24.921197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:24.921611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:24.923190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:24.934231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 250/488] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:24.935305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.007564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f27c22-364f-4c2a-858f-80808889da66 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.062580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.062580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.062580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.062580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.069288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:25.069288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.069288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa4ac5-7173-418a-89b4-47710dde1e19 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.069288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:25.080024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 239/489] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:25.081391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a34d-db60-403f-ab69-ce7312f9dffe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.081681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a34d-db60-403f-ab69-ce7312f9dffe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.081977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a34d-db60-403f-ab69-ce7312f9dffe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.090537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 240/490] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:25.090537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a34d-db60-403f-ab69-ce7312f9dffe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.094192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc397ec-3afb-4654-9cf2-19efd80bde9d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.147062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.147705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.148051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.148415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.149206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=bf272634ade542739c5cfe3139bb201a) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:25.149587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.151899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.157925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 251/491] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:24 2022] GET /identity/v3/role_assignments?user.id=bf272634ade542739c5cfe3139bb201a&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:25.158799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.162740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:25.240919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a34d-db60-403f-ab69-ce7312f9dffe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.298740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.300302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.300822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.301784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.316891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.319343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.330928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb7523-d23e-4585-87fb-cb63eb77af09 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b5bca4b6-152c-4f60-8cfc-22bccf75130d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:25.340749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:25.342340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:25.343337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:25.343337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:25.343337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:25.345880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 241/492] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:25.348139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.348588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.348994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.375562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:25.377218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:25.381938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.382607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.382896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.383179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.383917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=0f2a170d431b4524b18178757fa3d627) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:25.384231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.385738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.394304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 252/493] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/role_assignments?user.id=0f2a170d431b4524b18178757fa3d627&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:25.397856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.472381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:25.498954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:25.541809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707468-8a77-4873-b4d5-b7722d7873aa None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.548494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 242/494] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:25.597338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.598312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.598838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.599316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.600183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:25.600702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.602286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.616330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 253/495] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:25.622575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.726633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1e1b2-36f8-45eb-acfa-c602392304c2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.761329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:25.762287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:25.762800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:25.763277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:25.771340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:25.773036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:25.787506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:25.788573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:25.789494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:25.792651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 243/496] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:25.798319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.869984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:25.926853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:25.940372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef097ba-f0ad-46bd-a299-ceeeac11158b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:25.993168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.014835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.015458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.015737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.016009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.016751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:26.017087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.018494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.051784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc22d1f-04ef-4022-a9d7-a92572b56e13 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.054264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 254/497] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:26.063503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.108254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.108906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.109190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.109474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.129830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:26.144138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:26.145067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:26.145893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:26.152054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 244/498] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:26.156355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.218535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8cb3e-3327-418d-9c9e-c66a33227efc None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.254538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.255094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.255372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.255641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.271746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.273487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:26.306226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:26.307913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:26.309243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:26.309243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:26.309243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:26.311904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 255/499] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:26.317850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.326530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5493f-bb6c-4df7-927e-448c85bd619f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.332990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.361555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.362632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.362922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.363201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.364157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:26.364473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.365873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.375159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 245/500] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:26.381196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.402853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.497264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f569-398a-4f19-b547-67106b93507a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.542011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd32e04-aaf7-4b01-8395-67eaa32adac5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.557761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.558669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.559127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.559572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.560486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:26.561012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.562577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.578694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.579442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.579798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.580148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.590281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 256/501] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:26.594708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.596244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.598015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:26.616100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:26.617166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:26.618351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:26.618351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:26.618351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:26.618351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 246/502] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:26.624165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.640880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.769921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdda88-5ada-411e-a462-a5554fdb9a88 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.805075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.812521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb273-e1f6-468c-aced-6596039bbd18 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:26.841535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.842149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.842432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.842759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.843378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:26.846320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.846320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.870619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 247/503] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:26.877833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.879466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:26.880119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:26.880408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:26.880694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:26.895500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:26.897045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:26.916706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:26.917782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:26.918725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:26.926600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 257/504] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:26.927486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:26.978877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:26.990790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f5-dbd8-44c8-ba1d-ea6159db50a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:27.060386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:27.061003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.061284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.061726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.080499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:27.081987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:27.100115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:27.101234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:27.102213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 248/505] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:27.114110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:27.122975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8ff0e-d71c-46f8-8f09-3ec6e11ddba7 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:27.194670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:27.195306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.195591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.195864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.196586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:27.196893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:27.198969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:27.227445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 258/506] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:27.236199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:27.269622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:27.316450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83919d-4eeb-4c21-9d24-8bf47ce6d17d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:27.356183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:27.362823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3575eaf-c274-4be6-8414-c06942020dec None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:27.363279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:27.363892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.364442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.364718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.366353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:27.366699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:27.368141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8438b4-74b7-40a4-aa5a-7113eb04e0a2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:27.397034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 249/507] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:17:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:27.397034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195258-539d-41b6-a10e-a5375feb24a0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.397034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195258-539d-41b6-a10e-a5375feb24a0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.397034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195258-539d-41b6-a10e-a5375feb24a0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.397034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 250/508] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:27.403981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195258-539d-41b6-a10e-a5375feb24a0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:27.408750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:27.410021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.410537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.411120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.413378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=f025b86069754817868da5d0b19e3de0, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:27.415072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103943cd-a9f0-4c0a-928a-bde9e3d80961 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:27.485898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 259/509] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:27 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:27.487601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7032f2-caad-4603-b72f-a0256aacd31a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.488113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7032f2-caad-4603-b72f-a0256aacd31a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.488567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7032f2-caad-4603-b72f-a0256aacd31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.522547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ff7032f2-caad-4603-b72f-a0256aacd31a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:27.528360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ff7032f2-caad-4603-b72f-a0256aacd31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:27.559036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195258-539d-41b6-a10e-a5375feb24a0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:27.627556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:27.628505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.628993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.629902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.674321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 260/510] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:27.723798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:27.736110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:27.737698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:27.793710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dca441-2f0b-467b-a078-773f8262b811 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.794223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dca441-2f0b-467b-a078-773f8262b811 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.794694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dca441-2f0b-467b-a078-773f8262b811 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.796461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 261/511] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:27.802997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:27.803288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:27.803574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:27.839515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:27.840714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:27.842803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:27.849439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 251/512] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:27.852507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:27.969270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 262/513] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:27.986427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.000656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e48772-d282-4e1e-b1d8-88010bdd0f5d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.038112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.046360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.047031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.047314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.047593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.061434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.063178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:28.094003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:28.095054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:28.109473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:28.109473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:28.109473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.109473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 252/514] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:28.115540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.168213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de612552-72f4-43e8-87be-af3cef548c0b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.213700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.215238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.215987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.216435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.226232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.226993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dca92-f835-47a0-89bd-10fa7843a1f2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.236220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.237917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-d6ff4c6d-ad11-4ba2-8db0-6e144d59f9c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:28.258433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:28.259444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:28.260451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:28.260451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:28.260451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:28.266909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 263/515] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:28.268175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebafd-02ef-469e-9459-e0d11771adf5 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.268435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebafd-02ef-469e-9459-e0d11771adf5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.268714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ebafd-02ef-469e-9459-e0d11771adf5 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.270232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 264/516] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:28.272959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.273792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.274589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.275004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ebafd-02ef-469e-9459-e0d11771adf5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.275800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.276655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:28.277099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.278841 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca868f5f-a045-4a59-8a17-e962313f1a5f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.294471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 253/517] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:28.296674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.297105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.297519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.305237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.317228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:28.318442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:28.386279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ebafd-02ef-469e-9459-e0d11771adf5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.387084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 254/518] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:28.394719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.432753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.433425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.433951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.434230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.435173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:28.435598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.437499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.454667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 265/519] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:28.455589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.508178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd338e-0323-4c8d-b00f-7ebb87ca8209 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.509012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.546741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.553229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.554133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.554413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.554725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.555309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.557225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.576345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fddfe3-ae4e-4c5c-af5b-80b30cda242b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:28.576505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:28.578143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:28.579163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.580486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 255/520] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:28.581571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.606433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.607096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.607374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.607646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.616707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.618428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:28.633171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:28.634263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 266/521] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:28.639786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.700446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f7ee8-5fb6-406f-a03d-e8e8e8a7e6a7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.739531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.740145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.740430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.740709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.741353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.742868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.747024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc148bb-d1e2-4619-9de3-10d6f9362c66 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.758322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.780713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.781326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.781606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.781879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.782549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:28.782858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.784235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.788651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 267/522] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:28.789213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.810673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 256/523] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:17:28 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:28.810673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.879776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:28.933746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fec956-70ce-4e4f-97eb-4917983b33a2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.940279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c031b6-92d8-40d0-ae4d-75bdb7489a97 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:28.974459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.975132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.975417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.975690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.976298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:28.977677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:28.987747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:28.988846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:28.989686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 257/524] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:28.990998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:28.995536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:28.998239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:28.998239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:28.998239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:28.998738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:29.001937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:29.001937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:29.021956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 268/525] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:29.027347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:29.066536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:29.114009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329dae8-84d6-4420-a3a7-1a81c55dcfb5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:29.131289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:29.147971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad08ab-ad79-4239-b55a-7774b7e5e096 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:29.149318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:29.149931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:29.150264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:29.150538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:29.151263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:29.151564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:29.153036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:29.166536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 258/526] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:29.166536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:29.220414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:29.221531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:29.222010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:29.222579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:29.223420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] RBAC: Adding query filter params (name=network) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:29.223925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:29.225815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:29.232021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 269/527] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:29.239967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:29.277541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4f87c-ba8f-4684-88a9-b94db2b7206a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:29.315407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:29.316036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:29.316321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:29.316600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:29.322546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=7498ce3216d1411c961279f321418b7b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:29.322546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:29.322546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bab76-cefd-4035-8d38-89f541cd8683 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:29.326449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 259/528] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:29 2022] GET /identity/v3/role_assignments?user.id=7498ce3216d1411c961279f321418b7b&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:29.405038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:29.410331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8aaaa-4e3d-4e96-a19c-242ab22dccaa None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:29.451923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:29.453383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:29.453865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:29.454336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:29.455209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] RBAC: Adding query filter params (type=network) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:29.455721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:29.458577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d4336-05d2-46f5-a4f8-c88f231b8267 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:29.466805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 270/529] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:30.095137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286da2-fcbd-4040-bd18-3983e5146fab None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:30.095643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286da2-fcbd-4040-bd18-3983e5146fab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:30.095745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286da2-fcbd-4040-bd18-3983e5146fab None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:30.097227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 260/530] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:30.102825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:30.103113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:30.103438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:30.127655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:30.128842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:30.219761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 271/531] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:30.638338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286da2-fcbd-4040-bd18-3983e5146fab None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:30.686368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:30.740836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286da2-fcbd-4040-bd18-3983e5146fab None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:30.774883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:30.775543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:30.775821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:30.776092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:30.786374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:30.792110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:30.793067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:30.793923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:30.797937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 261/532] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:30.799435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:30.921623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:30.930398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696c1ce-5bbb-4843-9b7f-8fa3255ebbd0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:30.981080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:30.981704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:30.981991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:30.982268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:30.983025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:30.983338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:30.984828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:30.998182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 272/533] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:31.004014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:31.202754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3836b58-8e13-43b5-9ee6-38151351254f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:31.205889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:31.249570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:31.250198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.250532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.250811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.263138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:31.264615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2d6c704b-4582-4bf4-a877-7df936bf78fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:31.291631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:31.292653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:31.293558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:31.293558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:31.293558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:31.293558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 262/534] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:31.299621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:31.381925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5ccd-2c55-450a-b256-d59066ed16ad None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.382278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5ccd-2c55-450a-b256-d59066ed16ad None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.382552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5ccd-2c55-450a-b256-d59066ed16ad None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.384226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 263/535] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:31.394814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.395097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.395426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.430645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:31.433530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:31.465977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458de2-b614-4d4d-b448-6d879895386b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:31.536828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:31.537443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.537725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.538001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.538673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:31.538980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:31.540374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:31.565694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 273/536] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:31.567446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 264/537] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:31.575531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:31.596849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:31.699636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51c65-cdfe-4e14-82a8-2318f3589453 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:31.751338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:31.752325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.752807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.753301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.761199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:31.766950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-9eb8cbca-f886-448d-9985-33a86fee240e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:31.780852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:31.781902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:17:31.782873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:31.785379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 274/538] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:17:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:31.791547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:31.894312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:31.913938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2b512-0e09-4e85-babd-fe03fa1106ef None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:31.939991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ec98-8298-44f5-a361-9be1ce82e12f None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.941271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ec98-8298-44f5-a361-9be1ce82e12f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.941773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ec98-8298-44f5-a361-9be1ce82e12f None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.943690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 275/539] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:31.949050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.949675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.950148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.973363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:31.973944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:31.974057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:31.974255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:31.974604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:31.975291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:31.975591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:31.976370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:31.977030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.009136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 265/540] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:17:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:32.016930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.084407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 276/541] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 13:17:32.086555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.090942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.179812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5699993-0c96-4408-8d52-3b125f02f283 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.216788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.217398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2da75-6f11-4916-b290-f04920d13cc8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.217517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.217679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.218268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.235023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.251244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.251874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.252155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.252427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.263246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e859bffc-d789-4316-a03b-03076b0e1997 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:32.268101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:32.269205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:32.270063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 277/542] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:32.271259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29e7d-d883-4b17-a8f9-715738b1ed55 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.271259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29e7d-d883-4b17-a8f9-715738b1ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.271259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29e7d-d883-4b17-a8f9-715738b1ed55 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.272284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 278/543] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:32.277051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9096-aa6a-42e3-80c5-f2a614c86f0b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9096-aa6a-42e3-80c5-f2a614c86f0b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.277596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9096-aa6a-42e3-80c5-f2a614c86f0b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.279336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 279/544] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:32.282594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e9096-aa6a-42e3-80c5-f2a614c86f0b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.299692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.301135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c59b-2e33-487a-baa6-7443b5763393 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.347377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.392159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e9096-aa6a-42e3-80c5-f2a614c86f0b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.406796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 266/545] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 13:17:32.408105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.408389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.408668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.425477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:32.426694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:32.428563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.429754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.429754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.429754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.430347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:32.430683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.432066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f30cf3-176e-44d3-a471-b5e36bcd9e13 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.439873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 280/546] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:32.441038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.441328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.441611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.459416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:32.460597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:32.467344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.492916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 267/547] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:32.493715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.528569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 281/548] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 13:17:32.529750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.588481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.631222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732e5ba-f473-4604-beab-1505fa50a61d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.634376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b940ae-926d-40bd-99d3-7ef9329ebc9d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.668379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.669010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.669286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.669632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.679741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.682260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.682772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.683649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.684406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.685280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.686177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.687767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:32.699396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:32.700387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:32.701950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:32.701950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:32.701950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:32.701950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 282/549] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:32.701950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.777052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 268/550] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:32 2022] POST /identity/v3/services => generated 251 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:32.778901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.806014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8804-020d-4545-9e89-8b9583a59dbb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.818631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.825002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:32.856401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.857211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.857491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.858068 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.874705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.876181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:32.892060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:32.893486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:32.894389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:32.894389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:32.894389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:32.894389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 283/551] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:32.894389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:32.906119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7eb16-0183-43d1-b135-34cf67a694fe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:32.948892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:32.949663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:32.949858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:32.950085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:32.950763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:32.951069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:32.952654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:32.967183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 269/552] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:32.968129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.012690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf727-e5f9-40d9-aecc-f2a7d3c755d9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.050441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.051190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.051481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.051756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.063474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:33.066100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:33.069367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:33.070333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:33.071209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:33.075621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 284/553] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:33.075733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f50c13-350a-4a11-9f06-85a403e01180 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.076224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.095677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.113582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.114191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.114473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.114791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.115460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.116879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.176664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf853-9949-48d3-bd98-06fcaa12ec42 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.210335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.210981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.211263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.211536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.212173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:33.212476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.214406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.230575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 285/554] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:33.230575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.232515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 270/555] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:32 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:33.233354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.339747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.370619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc913a-f2b7-4a21-8c88-e8b98bc72e4f None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.384396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2c82-6bb5-4c5d-95ca-8d2758e176b1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.412829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.414453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.414966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.415688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.424899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.426945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.427638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.428249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.428532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.428809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.429483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:33.429784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.431175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.441064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 271/556] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:33.441879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:33.442368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:33.443259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:33.444167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:33.445035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 286/557] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:33.447637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.461691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.563057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d91d20-dd0e-47b7-95b6-781949d56a89 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.576949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45052391-08eb-47af-bc38-c8a98bb99447 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.610574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.611384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.612070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.612345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.619579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.620302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.620657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.620785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.621049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.622137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.624436 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.633035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.635138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:33.638385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:33.639603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:33.640798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:33.643001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 272/558] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:33.685502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:33.685676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:33.686821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:33.687948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:33.687948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:33.687948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:33.687948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 287/559] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:33.687948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.735424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.770785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07955b5b-662b-4224-ad92-d195fac3222a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.771253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c3c43-7b11-4334-95fe-0970e77162d3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.806407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.807180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.807567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.807942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.808936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:33.809383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.810955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.811815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.812670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.813090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.813503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.814440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:33.814916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.816476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:33.830658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 288/560] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:33.831659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.837246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 273/561] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:33.839018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:33.882862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.945122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f1cc-689a-42af-aa9b-8a68ca66c58b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.956142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694df03e-323c-4771-b28c-5dd281619a3a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:33.988283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:33.994349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.994871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:33.995192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.995542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.995882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.996108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:33.996537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:33.996818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:33.997464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:33.997768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:33.999731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.016316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.017815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.021807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 289/562] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:34.023352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:34.038894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:34.039899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:34.041275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:34.041275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:34.041275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:34.041275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 274/563] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:34.041275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.153967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcc5e6-c617-43ab-9605-67b3aa2e5bf7 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.161932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f3963-c744-4cd4-a05b-4754161c6805 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.163038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.201320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.202895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.203945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.204444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.205421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.205617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=f025b86069754817868da5d0b19e3de0) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:34.206086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.206201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.206615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.206981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.208411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.217921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.222714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.222869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 290/564] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/role_assignments?user.id=f025b86069754817868da5d0b19e3de0&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:34.223853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:34.246339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:34.248090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:34.249397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 275/565] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:34.253301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.363789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0fc6d-eb7c-4c2c-9bb9-33f432756d10 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.388697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.414320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.414320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.414320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.414320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.414756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:34.414756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.414756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.424873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 291/566] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:34.434117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.480279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.485211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0ae8c-0e19-48f8-b74c-992971bcaf57 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.522537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.523175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.523455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.523728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.524837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:34.524837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.526226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.545435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 276/567] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:34.551016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.563246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416626b2-fd32-412a-9418-b0641df5b3f3 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.602650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.604173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.604792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.605229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.612695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.614351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:34.627141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:34.628402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:34.629535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:34.631562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 292/568] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:34.637288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.657908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850c330-4a91-4d41-a73e-b593021652e1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.678939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.693608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.694345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.694532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.717462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.717462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.717462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:34.722101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:34.723159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:34.724051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:34.724051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:34.724051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:34.724051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 277/569] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:34.729485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.743627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.754537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2593bd-3de0-47ba-b32b-9adcf0f81011 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.792481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486888ff-7eae-411f-8363-61a29280689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.793096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486888ff-7eae-411f-8363-61a29280689b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.793382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486888ff-7eae-411f-8363-61a29280689b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.793757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486888ff-7eae-411f-8363-61a29280689b None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.794464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486888ff-7eae-411f-8363-61a29280689b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:34.794805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486888ff-7eae-411f-8363-61a29280689b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.796233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486888ff-7eae-411f-8363-61a29280689b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.808938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 293/570] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:34.814672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486888ff-7eae-411f-8363-61a29280689b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.830281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f18a5-7ffa-4eeb-98f7-ae3167e97f53 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:34.865754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:34.866365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:34.866679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:34.866951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:34.867594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:34.867894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:34.869299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:34.881782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 278/571] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:34.887891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:34.914864 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:34.973032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486888ff-7eae-411f-8363-61a29280689b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:35.050681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:35.051451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.051942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.052221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/612a5c64ff3d4ec1826cf002bec59e26/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.068863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=612a5c64ff3d4ec1826cf002bec59e26, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:35.070315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfa302-a500-4585-9dd2-4b08f735a5f3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:35.082438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd8e7-c9a5-4a96-b00e-05f6fc5d63ec None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:35.119870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:35.120511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.120792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.121067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.128025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:35.129487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7478960c-8e34-4ecf-9b38-327942593449 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:35.141189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:35.143295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 279/572] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31992f18-6fe2-4992-bb64-1cfb22f59feb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.145895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31992f18-6fe2-4992-bb64-1cfb22f59feb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.147435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31992f18-6fe2-4992-bb64-1cfb22f59feb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.154617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 280/573] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:35.154617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31992f18-6fe2-4992-bb64-1cfb22f59feb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:35.156333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:35.162316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 294/574] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:34 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/612a5c64ff3d4ec1826cf002bec59e26/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:17:35.163910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.164301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.164684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.195711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:35.198170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:35.305277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 295/575] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:35.326945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:35.366385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31992f18-6fe2-4992-bb64-1cfb22f59feb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:35.410623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:35.411728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.412240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.412713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.413685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:35.414196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:35.416464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:35.429649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 281/576] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:35.438370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:35.589331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5384e6-d3b9-4e42-a92c-ee14fd6d2457 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:35.633305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:35.639795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:35.640570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.640854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.641131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/7498ce3216d1411c961279f321418b7b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.642272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:35.643221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=7498ce3216d1411c961279f321418b7b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:35.644815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747dd26-854b-42dd-b512-6c5bab897cc1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:35.757490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226c2e6-95f2-4e76-bf72-5099077d888e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:35.770738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 296/577] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:35 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/7498ce3216d1411c961279f321418b7b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:35.773828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ae2d3-485b-4f35-9fab-c6987f8b5fc3 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.774346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ae2d3-485b-4f35-9fab-c6987f8b5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.774890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ae2d3-485b-4f35-9fab-c6987f8b5fc3 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.777778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 297/578] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:35.783811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.784132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.784362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.807473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:35.808485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:35.809081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:35.810220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:35.811125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:35.812875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:35.815589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:35.825125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:35.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:35.827142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:35.828161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:35.828256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 282/579] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Oct 19 13:17:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:35.831525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:35.833682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:35.969417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 298/580] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:36.006433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ba4e-ed9f-4cce-9ed7-fbfc802e39a3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.045208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.045821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.046100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.046371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.047029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:36.047329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.048684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.053624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 283/581] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:36.061755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.166853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945e0f-2ad4-4c2a-a9d5-b376b9291cf9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.192803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:36.212094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.213497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.213952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.214401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.215266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:36.215808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.217390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.226845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 299/582] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:36.232747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.290336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.330856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:36.427963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf260f0-4382-48eb-8f36-5b6daaf51b6a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.428650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6211d8-faa0-4d8e-8f2e-f59a891a2d1e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.468575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.469215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.469504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.469781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] PATH_INFO: `/v3/services/network` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.472528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.472528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:36.486140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:36.487310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:36.488229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:36.488980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 300/583] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:36.495526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.512105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.512732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.513012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.513284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.513990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] RBAC: Adding query filter params (interface=public, service_id=e29eb56f7c9340c0b94fd1b6e8a6d871, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:36.514295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.515678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.531874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 284/584] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/endpoints?service_id=e29eb56f7c9340c0b94fd1b6e8a6d871&interface=public®ion_id=RegionOne => generated 194 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:36.537009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.555955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:36.614105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360820b1-b777-4b7f-9c98-364df1d37e7d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.656300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] RBAC: Adding query filter params (name=network) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.660671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.665311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 301/585] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:36.672174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.788581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7f965-8bf8-4818-904f-1239181d6401 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.788887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d171a-5c75-4ba3-9ee5-d8eb9686d95c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:36.806857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:36.828616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.829364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.829712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.835369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.835369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] RBAC: Adding query filter params (type=network) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:36.835369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.835369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.839620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 302/586] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:36.849206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:36.851485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:36.860822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:36.861476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:36.861787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:36.862049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:36.863222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:36.865764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:36.871997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 285/587] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/services => generated 772 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:36.972498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56441efa-b724-444c-9fa8-9af3bd375ece None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:37.010374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:37.011045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.011326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.011601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.012323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] RBAC: Adding query filter params (interface=public, service_id=356979bfd85843d8a79e17979c252f1c, region_id=RegionOne) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:37.012634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:37.014019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe240-836b-4eef-8c76-d7bb46de18dc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:37.022634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 303/588] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:36 2022] GET /identity/v3/endpoints?service_id=356979bfd85843d8a79e17979c252f1c&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:37.028432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:37.172417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aeb80-fce0-4bc2-8f48-fa90e4a387de None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:37.176530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5416a2-e4d9-410d-bbe2-a5118f73931d None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.177001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5416a2-e4d9-410d-bbe2-a5118f73931d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.177092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5416a2-e4d9-410d-bbe2-a5118f73931d None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.178634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 304/589] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:37.184703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.184992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.185259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.208490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:37.210156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:37.215252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:37.216145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.216641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.217114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.218053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:37.219653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:37.225897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 286/590] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:37 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:37.249983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:37.286684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 305/591] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:37.590522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:37.654692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:37.703446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a84b9-5715-4558-bb93-9626fa4ab6c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:37.748092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:37.748728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.749006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.749277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.749853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:37.751228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-91c80612-44f3-4f78-ad5d-0c66106c2d68 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:37.764571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:37.765538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:37.766450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 287/592] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Oct 19 13:17:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:37.771711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:37.779034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad96c-2479-4631-b846-408fc02b107b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.779320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad96c-2479-4631-b846-408fc02b107b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.779594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aad96c-2479-4631-b846-408fc02b107b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.781181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 288/593] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:37.791469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:37.791774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:37.792020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:37.822935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:37.824132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:37.896236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 289/594] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:37.958308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af0a98-f7ec-45b1-b592-5d7ae9890759 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:38.019230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:38.019872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.020164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.020447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.021099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:38.021409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:38.022826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:38.028459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 306/595] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:38.034360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:38.111269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:38.178903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca492ce-2e33-4650-987d-3b461c1296cb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:38.230367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:38.231012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.231297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.231576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.232220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:38.232529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:38.233905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:38.243359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 290/596] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:38.254755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:38.310284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:38.411566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa367b-b592-4736-b7ed-61d2f208c5d6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:38.459095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:38.459679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.459961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.460237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.479621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-079e7d60-ef03-497f-a8fa-03052cc81db8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:38.484711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:38.485776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:38.487366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 307/597] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:38.491892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:38.537867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7fcac-1dab-4065-963a-8abfe2c90708 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.538230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7fcac-1dab-4065-963a-8abfe2c90708 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.538435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7fcac-1dab-4065-963a-8abfe2c90708 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.540169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 308/598] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:38.546993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.547644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.548122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.583005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:38.584696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:38.649367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26d9c-837d-46bd-83a5-4392ab6179d0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:38.672367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:38.696475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 309/599] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:38.702229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:38.703127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.703626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.704146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.705007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:38.705539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:38.707203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:38.724669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 291/600] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:38.730467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:38.769075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:38.859968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8864048-6617-4e2b-9060-0547259c58d6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:38.902283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:38.903112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:38.903399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:38.903679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:38.917728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:38.919272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:38.938884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:38.939855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:38.941405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:38.941405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:38.941405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:38.941405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 310/601] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:38.945718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:38.983688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.118480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.128877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df8d4-5b5c-4b23-a127-eb746a46a8d7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.161382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5877c22-5b13-4e91-b800-d9c5b2eb5d04 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.190770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.191383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.191659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.191928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.192573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:39.192869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:39.194247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:39.214174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 292/602] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:39.219671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.235828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.236410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.236696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.236976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.267407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:39.270365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:39.273656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:39.274638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:17:39.275543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:39.283240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 311/603] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:39.289202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.345161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f2023-0c8c-44a5-888c-b24a9401812f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.382967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.383951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.384996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.385456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.393266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:39.395022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:39.407225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:39.408215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:39.409191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:39.411992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 293/604] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:39.418858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.517367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aeae06-115d-4109-9365-88b4b0244e74 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.580574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.581576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.582141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.582659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.583554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:39.584317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:39.586782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:39.598848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.606871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.611881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 312/605] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:39.620108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb6ff-05b0-4cb0-b483-00d182e0f4d2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.620258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.696550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.697930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.698446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.698967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.699916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:39.702186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:39.703868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:39.734489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 294/606] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:39.742533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.797853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530bd0d-12a7-40a5-a697-948f4e638e6d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.834635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.835630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.836118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.836625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.847673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:39.849423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-5af8204a-af35-499f-9673-597e4be873ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:39.865528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:39.866546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:39.867482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:39.867482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:39.867482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:39.870094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 313/607] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:39.871576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd53c6-3e3b-4240-8952-45e723511aba None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.872166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd53c6-3e3b-4240-8952-45e723511aba None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.872653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd53c6-3e3b-4240-8952-45e723511aba None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.878252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c5904-123e-428a-a601-4badba6cf0f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:39.878498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 314/608] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:39.879499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd53c6-3e3b-4240-8952-45e723511aba None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:39.920145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:39.920759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:39.921029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:39.921486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:39.935708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.948968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:39.999585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd53c6-3e3b-4240-8952-45e723511aba None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.009406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.011060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.042294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.043167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.043703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.044164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.045071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:40.045548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.047157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638ea9f-eb06-4b22-b595-5d3659f3a7a9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.060346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 315/609] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:40.061887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362735d-519e-48a1-ba45-ec2ba0c89738 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.062349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362735d-519e-48a1-ba45-ec2ba0c89738 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.062951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362735d-519e-48a1-ba45-ec2ba0c89738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.082118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6362735d-519e-48a1-ba45-ec2ba0c89738 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:40.084062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6362735d-519e-48a1-ba45-ec2ba0c89738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:40.122906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 295/610] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:17:40.123877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 316/611] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a301d-5d8a-4ca3-be8f-7bc626aeb000 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a301d-5d8a-4ca3-be8f-7bc626aeb000 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a301d-5d8a-4ca3-be8f-7bc626aeb000 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 317/612] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:40.162178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a301d-5d8a-4ca3-be8f-7bc626aeb000 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.196393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.213022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.261259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4708d-fcbb-4ec9-a69f-202b6b3dd91f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.277769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a301d-5d8a-4ca3-be8f-7bc626aeb000 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.306644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.307401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.307924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.308212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.317027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.317601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.317886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.317996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.318260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.319607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.329677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.331177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b40d4eb1-c6d9-4b1a-be6d-02277fde0b79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:40.333613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:40.334847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 13:17:40.335843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:40.341118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 296/613] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:40.342934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.343228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.343507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:40.349718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:40.351980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:40.354709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:40.354709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:40.354709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:40.354709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 318/614] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:40.354709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.372437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:40.373883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:40.452243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.461089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb844-d646-4c33-84c0-565d882a9bf8 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.474530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 297/615] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:40.474530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.514477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.515824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.516259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.516551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.516827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.517852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:40.518218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.519863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.544187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 319/616] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:40.546263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.576782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e83e9-4903-4f57-9c76-cef43244af97 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.613428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.614197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.614829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.615114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.615771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:40.616073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.617442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.630266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 298/617] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:40.631347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.670207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbd4c9-399a-49b0-8397-553fb7c68b36 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.707938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.709219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.709716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.710194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.711024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.713184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:40.722766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:40.723823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 13:17:40.724695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:40.727424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 320/618] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:40.728527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.746694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.752475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912884f1-efe2-41ef-9e61-b75266c62f5a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.754073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.787644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.788247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.788522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.788791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.835741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49d980-61b7-4180-9b2d-581c5415488e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:40.856710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.858189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83027a91-293c-4189-87bc-d7b5ef799c56 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.867647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:40.868277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.868556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.869340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.875786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:40.877219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:40.888722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:40.889635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 321/619] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:40.890995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.958232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 299/620] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:17:40.959852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0249dd-4a94-48ae-99ef-7995ba81aa62 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:40.960139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0249dd-4a94-48ae-99ef-7995ba81aa62 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:40.960415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0249dd-4a94-48ae-99ef-7995ba81aa62 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:40.966408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 300/621] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:40.966408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0249dd-4a94-48ae-99ef-7995ba81aa62 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:40.967472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.968835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:40.988781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaac78-3f54-4861-bafc-bdc34f111644 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.024234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.025339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.025629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.025899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.026535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.026844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.028207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.032538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 322/622] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.033315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.068789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0249dd-4a94-48ae-99ef-7995ba81aa62 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.102774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.104085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.104529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.104971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.105631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.107158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:41.115906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:41.116956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:41.117859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 301/623] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:40 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:41.118783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.139873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff69ed-c356-458b-870e-9141f83ce772 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.173024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.174032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.174529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.174823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.175517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.175824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.177255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37027b-b76a-4d41-a04c-d8cedb95a9dd None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.185255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.187776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.190135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 323/624] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.192799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.192799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.192799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.209324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:41.210527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:41.223208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f68d-0108-4eae-86b4-ad1eb3afe3fb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.256878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.257499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.257777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.258059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.271527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.272956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.276758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 324/625] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:41.277584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:41.286820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:41.287786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:41.288704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:41.288704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:41.288704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:41.288704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 302/626] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:41.288969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.389938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145f700-b2d0-4cbb-9563-e13aff1a9db6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.395542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6820-e99e-4701-a2fc-c2a6e001027c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.413930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.418298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.426152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.426788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.427455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.427455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.428024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.428323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.429691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.432433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.433249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.433702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.434147 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.434968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 325/627] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.435141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] RBAC: Adding query filter params (name=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.435611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.435747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.437308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.443588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 303/628] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.444428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.546902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28259c38-724c-4299-8896-ddedbe02c9ea None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.559810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec419d7-7bc8-4851-8633-8ebfc40aca76 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.580653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.581262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.581538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.581805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.582525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=612a5c64ff3d4ec1826cf002bec59e26) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.582877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.584544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.590800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 326/629] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/role_assignments?user.id=612a5c64ff3d4ec1826cf002bec59e26&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.591542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.594456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.595100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.595379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.595650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.596299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.596602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.598004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.612418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 304/630] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.613307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.666682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.667292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.720612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210a563-4f61-4e99-be98-9b1385481edb None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.730221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c8cce-eef6-4551-9189-7a477e767cb2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.759835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.760455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.760740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.761020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.761774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.763116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.769871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.770478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.770785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.771055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.771678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:41.771980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.773330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:41.779440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 305/631] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:41.780312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.869842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 327/632] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:17:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:41.870721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:41.895344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8914-a69c-465d-a698-df4d23a19bd8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:41.906001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:41.952674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:41.954149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:41.954542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:41.954966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:41.956383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:41.958040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:42.013836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8848d6-13d1-4e68-bc90-605e57cfcba1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:42.032187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:42.034946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 306/633] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:41 2022] POST /identity/v3/services => generated 269 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:42.036395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:42.085260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:42.085977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:42.086260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:42.086679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:42.097122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:42.099573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:42.117434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:42.118596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:42.119561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 328/634] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:42.124804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:42.233388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8cf33b-06ae-4e5e-a4b5-1241c36d1898 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:42.257105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d2ca-21f1-4c59-a588-5fefcdebf186 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:42.277434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:42.278188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:42.278611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:42.278984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:42.279750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:42.281232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff61d39-6b92-4c12-a944-718aa862d588 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:42.301085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:42.304354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:42.304354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:42.304354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:42.304354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:42.304354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:42.304731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:42.319949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 329/635] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:42.325287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:42.358065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 307/636] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:17:42 2022] POST /identity/v3/endpoints => generated 337 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:42.364591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:42.462183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a88cfc-7942-4d48-bb40-df893752b4f6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:42.526756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:42.527752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:42.528245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:42.528741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:42.529709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=7498ce3216d1411c961279f321418b7b) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:42.530252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:42.531976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038c53a-91d3-4e04-9ba1-8e73c9ade716 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:42.539125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 330/637] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:42 2022] GET /identity/v3/role_assignments?user.id=7498ce3216d1411c961279f321418b7b&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:43.855044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689df7-0658-4ff2-9ce2-44045c8fd44b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:43.855564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689df7-0658-4ff2-9ce2-44045c8fd44b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:43.855668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a689df7-0658-4ff2-9ce2-44045c8fd44b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:43.857152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 308/638] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:43.862467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:43.862887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:43.863093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:43.883153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:43.884908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:43.955414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 331/639] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:43 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:44.228886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689df7-0658-4ff2-9ce2-44045c8fd44b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.249119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.341433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a689df7-0658-4ff2-9ce2-44045c8fd44b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:44.381832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:44.382912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.383419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.383897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.394839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cc8b7c71-2d29-47f2-a7ff-9641b5a21249 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:44.399994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:44.400964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:17:44.401812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:44.404145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 309/640] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:44.410536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.420462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993e39-8581-406e-a025-d9ff129922cc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.420736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993e39-8581-406e-a025-d9ff129922cc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.421014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993e39-8581-406e-a025-d9ff129922cc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.422928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 310/641] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:44.429167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ba485-2c87-4eba-baaf-a530459b342d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.429623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ba485-2c87-4eba-baaf-a530459b342d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.430067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ba485-2c87-4eba-baaf-a530459b342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.430690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.460318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-570ba485-2c87-4eba-baaf-a530459b342d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:44.461538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-570ba485-2c87-4eba-baaf-a530459b342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:44.528660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309ae74-bdff-4071-81f0-960c525e1594 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:44.538222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 311/642] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:44 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:44.540366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a5b-10d2-40e4-ad05-0bee467290f6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.540825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a5b-10d2-40e4-ad05-0bee467290f6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.541269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50925a5b-10d2-40e4-ad05-0bee467290f6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.543300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 312/643] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:44.548391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.548675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.548949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.567430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:44.568653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:44.570791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.579544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:44.580572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.585822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.585822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.585822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:44.585822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:44.585822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:44.596238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 332/644] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:44.601953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.648226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 313/645] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:44 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:44.700471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.720299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9d058-05fe-47d1-89f2-c8374645d897 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:44.756117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:44.757074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.757538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.758035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.768939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:44.770647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:44.785806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:44.786773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:44.787688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:44.787688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:44.787688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:44.789486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 333/646] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:44.795819 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.825765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.930389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905087f-50ff-4ca8-9a97-9fcdd611ec3e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:44.936348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1fd87-e81e-4851-90e0-4e9042714de3 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:44.965348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:44.965530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.965959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.966351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.966583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.967234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:44.967545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:44.968939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f0081-b3ab-4505-869d-4c49f291dedb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:44.970304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:44.970953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.971236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.971509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.972081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:44.973419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:44.975455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:44.981435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 314/647] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:44.982584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c7d0-8103-43aa-8c03-022a8db5e73b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:44.982704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:44.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:44.987937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:44.989191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c7d0-8103-43aa-8c03-022a8db5e73b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:44.989191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c7d0-8103-43aa-8c03-022a8db5e73b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:44.989355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 334/648] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:44.989355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:44.989491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 315/649] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:44.989491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c7d0-8103-43aa-8c03-022a8db5e73b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.102584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c7d0-8103-43aa-8c03-022a8db5e73b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.105291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ea6f-8aa7-4025-997d-d9edf646ac1c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.139865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.140498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.140783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.141059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.142312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.142993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.143279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.143555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.144144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.145525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.148410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.149854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f4938b2e-ac2d-4b95-a890-a5c3e00271ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:45.154465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:45.155607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:45.159541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 335/650] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:45.160350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.160350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.160350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:45.161531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:45.162433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:45.163452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 316/651] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:17:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:45.164051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.185699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:45.186937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:45.227051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.245195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.256087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 336/652] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:45 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:45.256949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.276112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047757cb-1a9a-49d7-a2b3-e27cf7bac571 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.318981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.319587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.320074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.320364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.320999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.321299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.322724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.327811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 317/653] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.328640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.364789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04cb16-edc8-44fa-8ee8-e8ad1a88d2eb None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.399644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.400872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.401158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.401428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.402056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.402357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.403775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.408847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 337/654] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.409456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.450071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35e703-db71-43dc-abeb-5a1e2f2e5cb4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.469082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.473437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.488542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.489154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.489432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.489701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.490542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.490859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.492335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.511658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 318/655] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.512318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.519830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d3be-725e-4406-a5d7-8c57199e69f1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.555471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.556599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.556881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.557156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.557784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.558087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.559490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.563928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 338/656] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.564700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.623389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7e201-fb2c-40d1-b492-56af52c4367c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.662264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.662911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.663205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.663469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.664107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.664410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.666639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.672837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 319/657] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.673701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.687238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25070709-1eb7-44b8-8492-c594333a3c49 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.706536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.724472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.725187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.725466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.725740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.727622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.738564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.740258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:45.757304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:45.760365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:45.761299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:45.761299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:45.761299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:45.761299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 339/658] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:45.761299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.794539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439eaa7-e04e-49f3-83ab-1985fe517fab None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.829404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.830284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.830744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.831170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.831899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.834361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:45.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:45.846710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:45.847752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 320/659] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:45.848603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.872204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3998a29-7dab-47b6-8a91-ef185fdf83fd None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.906958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:45.907574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:45.907926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:45.908266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:45.909089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] RBAC: Adding query filter params (interface=public, service_id=df8f13353638454582c3b0c4434fc50f, region_id=RegionOne) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:45.909517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:45.910919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:45.917185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 340/660] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/endpoints?service_id=df8f13353638454582c3b0c4434fc50f&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:45.918333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:45.959159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:45.965844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b131a-8606-4f02-812a-62ec3de9c504 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:45.975706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:46.002803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.003420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.003696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.003969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.004610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:46.004910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.006271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.023908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 321/661] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:46.023908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:46.034585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0393a-b5b9-404d-848b-d6a5896e8b40 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.072793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.073410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.073680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.073950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.074619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:46.074922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.076299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.085692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 341/662] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:46.085692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:46.139726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007452-7061-482c-bac0-a82797a90f17 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.174478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.175035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.175618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.175895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.176498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.178295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.184164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 322/663] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:46 2022] GET /identity/v3/services => generated 1029 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:17:46.194930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:46.198815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:46.203241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b7343-c120-4fd4-b78e-198f792cc954 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.232738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:46.253482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.254225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.254540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.254816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.266243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.268244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:46.290288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:46.291942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:46.292863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 342/664] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:46.299953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:46.331824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05b60f-4961-48e7-a58e-2632c39a5d52 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.376387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.377016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.377319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.377575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.378216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:46.378539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.380607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.387211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 323/665] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:46.430802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f61f-e64e-4765-a46e-bbc099eb80d4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.471244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.472254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.472694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.473182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.474149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:46.474666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.476366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706897f2-d211-4119-a172-1f06c16580d2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.492362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 343/666] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:46.498814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:46.602048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:46.608378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d859c-3485-4977-a894-c109a6bdfb6c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:46.642877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:46.643546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:46.643823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:46.644093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/612a5c64ff3d4ec1826cf002bec59e26/roles/f81416c420b3420c85b79049156eaf8f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:46.646462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=612a5c64ff3d4ec1826cf002bec59e26, role_id=f81416c420b3420c85b79049156eaf8f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:46.648091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e6151-9d10-4345-99e5-26c02ee8a467 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:46.728021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 324/667] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:17:46 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/612a5c64ff3d4ec1826cf002bec59e26/roles/f81416c420b3420c85b79049156eaf8f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:17:47.793316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe35a3-74a2-431b-843a-f8ff8053c6dc None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:47.793838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe35a3-74a2-431b-843a-f8ff8053c6dc None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:47.793838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe35a3-74a2-431b-843a-f8ff8053c6dc None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:47.795348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 344/668] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:47.800724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d626-e7a0-47d0-8cf0-16eea2f3a46f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:47.801509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d626-e7a0-47d0-8cf0-16eea2f3a46f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:47.801997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1d626-e7a0-47d0-8cf0-16eea2f3a46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:47.821379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-77b1d626-e7a0-47d0-8cf0-16eea2f3a46f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:47.822642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-77b1d626-e7a0-47d0-8cf0-16eea2f3a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:47.891392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 325/669] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:48.178152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe35a3-74a2-431b-843a-f8ff8053c6dc None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:48.199184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:48.292097 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe35a3-74a2-431b-843a-f8ff8053c6dc None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:48.327628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:48.328235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.328517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.328787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.329447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:48.330838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254fad6-4463-43fc-b47a-6a1f16ac2a8d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:48.442353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 345/670] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:48 2022] POST /identity/v3/services => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:48.480075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192321b3-69a5-451c-8071-914476bd9053 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.480392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192321b3-69a5-451c-8071-914476bd9053 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.480637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192321b3-69a5-451c-8071-914476bd9053 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.482492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 326/671] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:48.488224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357fe-1be3-4ec2-8671-545ac4bcb772 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.488532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357fe-1be3-4ec2-8671-545ac4bcb772 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.488801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357fe-1be3-4ec2-8671-545ac4bcb772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.509612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e15357fe-1be3-4ec2-8671-545ac4bcb772 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:48.510952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e15357fe-1be3-4ec2-8671-545ac4bcb772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:48.596821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 346/672] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:48.641550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b009d-c608-461c-9558-d132dd814342 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.641852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b009d-c608-461c-9558-d132dd814342 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.642122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b009d-c608-461c-9558-d132dd814342 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.643548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 327/673] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:48.648867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6c6-41fe-4d1c-b997-9e39625e2143 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.649165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6c6-41fe-4d1c-b997-9e39625e2143 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.649442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863b6c6-41fe-4d1c-b997-9e39625e2143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.657435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:48.669522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9863b6c6-41fe-4d1c-b997-9e39625e2143 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:48.670747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9863b6c6-41fe-4d1c-b997-9e39625e2143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:48.744417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 347/674] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:48.872888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b009d-c608-461c-9558-d132dd814342 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:48.922166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b746-db37-49de-9f06-5bcfdbfdd9ab None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.922713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b746-db37-49de-9f06-5bcfdbfdd9ab None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.922829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0b746-db37-49de-9f06-5bcfdbfdd9ab None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.924345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 348/675] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:48.930388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:48.930680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:48.931525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:48.956958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:48.958471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:48.965889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:48.983844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b009d-c608-461c-9558-d132dd814342 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.034830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.040048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.040048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.040048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.040048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.040048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.045951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 349/676] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:49.046913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:49.048765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:49.049895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 13:17:49.050964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:49.058261 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 328/677] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Oct 19 13:17:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:49.059399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.118382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:49.158876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d887f-092b-4f97-b4eb-f5cd49e0fa49 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.189605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0db59-9966-4437-b3a9-1d89c46f1edf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.199863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.200483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.200764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.201038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.202067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.203184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.229485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.230194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.230575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.230927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.231633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:49.232005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.233523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.238640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 329/678] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:49.243992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.260379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:49.306802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 350/679] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:49 2022] POST /identity/v3/services => generated 241 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:49.347143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.364748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9996852-4278-4c80-acf4-1d439b489844 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.415894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.416561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.416842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.417116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.417772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:49.418134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.419808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.426452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 330/680] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:49.431851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.472389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9769f6-5ead-41a1-9159-ba6796f46fed None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.499446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:49.513437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.514359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.514844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.515298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.523276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:49.529662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:17:49.535092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:49.536029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:17:49.536882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:49.539417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 351/681] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:49.546492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.556782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b39c7-a39b-4a19-9f13-1a4e1b50c0be None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.594126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.595078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.595541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.595993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.596864 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.598453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.661420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c104-e167-451f-982e-75217e5813e2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.679423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 331/682] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:17:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:49.725117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.725774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.726059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.726335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.727087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:49.727396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.728841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.738891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 352/683] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:49.745358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.858354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7692-6774-4d8f-bbe4-915a79852dbc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:49.890006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:49.897664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:49.898340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:49.898667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:49.898901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:49.909826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:49.911306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:49.925941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:49.926947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:49.927894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:49.927894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:49.927894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:49.927894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 332/684] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:49.935402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:49.986741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:50.058144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f455-a343-4714-86f9-f678ddd5c750 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:50.094806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:50.095428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.095712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.095995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.096706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:50.097015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:50.098465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:50.113417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 353/685] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:50.119640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:50.224331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5109e-683c-4472-ae38-d459afcf6c8c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:50.262438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:50.263123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.263406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.263681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.272622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:50.274086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:17:50.286566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:50.287621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:50.288669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 333/686] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:17:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:50.294305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:50.404630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb316-f7a5-4b14-b682-bd3e895e427c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:50.406616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:50.440638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:50.441416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.441791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.442168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.443040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:50.443452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:50.445043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af631fba-02ec-4f35-bb06-950c35f5f74d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:50.464749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 354/687] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Oct 19 13:17:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:50.471831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:50.517657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:50.578592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9cba-b96b-4130-a07d-8fdeabbd5dbe None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:50.616868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:50.617524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.617808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.618083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.698609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:50.700431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:50.717967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5474a-415b-487c-9b5a-b680f8ffd941 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.718701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5474a-415b-487c-9b5a-b680f8ffd941 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.719171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5474a-415b-487c-9b5a-b680f8ffd941 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.721458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 355/688] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:50.727515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.728581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.729306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.748816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:50.754482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:50.756355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:50.819027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 334/689] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:17:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:17:50.824258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:50.841795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 356/690] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:50.924905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1dae0-bfe6-4a5c-a594-549763917d33 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:50.959900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:50.960596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:50.960977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:50.961349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:50.972040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:50.973583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:50.980334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:17:50.988209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:50.989379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:17:50.990552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:17:50.990552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:17:50.990552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:50.991906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 335/691] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:50.997493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.108306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.118642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b0403-a558-49fd-a14e-2883b9dab7cb None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.133744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.156746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.157799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.158433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.159171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.159970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.160425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.161963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.174376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 357/692] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:51.179667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.265448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baae28f-6944-4b1b-a0ae-56053c4078ae None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.284557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f864b9f-b51f-46d3-adf6-653dec9e4023 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.306240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.307282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.307766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.308236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.309029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.311561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.320966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.322159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.322445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.322748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:51.324478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:51.325472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:51.326340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:51.329170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 336/693] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:51.334592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.335135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.336063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:51.348311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:51.349251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:51.350143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:51.354259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 358/694] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:51.355150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.368907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.419980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.455054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ad0f-0f20-4879-979c-8c970fa53767 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.483443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c22f4-7273-42ca-b3a3-7043f1dc12cd None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.496706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.497336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.497623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.498394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.498666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.498973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.500409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.511576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 337/695] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:51.512975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.534275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.535607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.536112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.536611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.537503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.538537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.540461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.558651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 359/696] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:51.564603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.636880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0807-e8c5-4340-af26-f5636868557a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.643643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.662624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.673993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.674567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115b431-c04b-4c6d-9c3d-f7b11b62da82 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.674705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.674922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.675198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.675824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.676131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.677501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735858-165f-4809-b86a-dd1d91754888 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.682531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 338/697] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:51.688161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c4d9-5aa2-47d4-9cef-caf13536af19 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.688460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c4d9-5aa2-47d4-9cef-caf13536af19 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.688744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c4d9-5aa2-47d4-9cef-caf13536af19 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.690174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 339/698] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:51.695300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510c4d9-5aa2-47d4-9cef-caf13536af19 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.714968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.715577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.715863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.716129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.716853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] RBAC: Adding query filter params (role.id=f81416c420b3420c85b79049156eaf8f, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=612a5c64ff3d4ec1826cf002bec59e26) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.717159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.718605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6201de-c5f4-44fe-833b-e89557968584 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.734545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 360/699] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/role_assignments?user.id=612a5c64ff3d4ec1826cf002bec59e26&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=f81416c420b3420c85b79049156eaf8f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:51.734545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.734545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.734545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.752596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:51.754039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:51.828325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510c4d9-5aa2-47d4-9cef-caf13536af19 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:51.831102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 361/700] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:51.881039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:51.881692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.881978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.882256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.883050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] RBAC: Adding query filter params (interface=public, service_id=26abf7ca886943ceb1b4252f43935a7a, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:51.883367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:51.885372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afa42c-856d-4a0d-a91e-5176a5c8a4b5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:51.892428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 340/701] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/endpoints?service_id=26abf7ca886943ceb1b4252f43935a7a&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:51.899003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:51.970428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:51.984731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3294-e05f-4fd4-b32e-c3046daaa5b3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.985257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3294-e05f-4fd4-b32e-c3046daaa5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.985257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb3294-e05f-4fd4-b32e-c3046daaa5b3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:51.989502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 341/702] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:51.993778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:51.994349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:51.994876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.017484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b0a69-264c-4154-beef-f852e198a9bf None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.025170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:52.026635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:52.057307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.058482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.059296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.060049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.061055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.062770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.078613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 362/703] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:51 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:17:52.084331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.107273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.109922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 342/704] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:52.226724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914636-004a-4207-865f-f89f26b520e8 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.256408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.264933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.265580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.268811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.268811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.268811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.268811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:52.277329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:52.278334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.279806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 363/705] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:52.286961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.394735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.397872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb5ed2-203b-4f3d-86e4-ec4068da2f95 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.440856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.441492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.442035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.442321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.442995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:52.443304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.444673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.450372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 343/706] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:52.456570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.507790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.516757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53922e-1130-49f6-83eb-3ffad13b60c9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.569365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b202-233d-48d7-a9be-4ad3bee0e0c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.570183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.571415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.572809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.573341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.574141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.575815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:52.589292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:52.590383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:52.591616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:52.595104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 364/707] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:52.600862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.608446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.608761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.609044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.609318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.609956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:52.610260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.611693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.622440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 344/708] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:52.623530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.679127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.734326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7311d-da82-4c46-973e-362c13b507c3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.744602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.747236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36806d7e-5250-4717-a945-0a3a6fecd080 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.773370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.775333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.776198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.776198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.777089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] RBAC: Adding query filter params (interface=public, service_id=88e1e891d4c44597b9319fd04495c0bf, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:52.777865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.779447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.791046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 345/709] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/endpoints?service_id=88e1e891d4c44597b9319fd04495c0bf&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:52.792556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.793954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.794810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.795296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.796009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.797737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:52.798200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.799782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.805834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 365/710] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:52.811389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:52.909986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad79b-9448-4930-82a1-46c88c7b8800 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.945942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.946754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:52.947137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:52.947512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:52.948367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:52.949821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5944a6e-48bf-48af-a75e-02d0168509d7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:52.950056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746a454-fbaf-416c-a168-9e3ac1ed56d7 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:52.958822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 346/711] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:17:52.978741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:52.997156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:52.998095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:53.002698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:53.003148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:53.003883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:53.004708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:53.005617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35f338-32d2-4558-a3fc-9113f0b8a674 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:53.012792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 366/712] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:17:54.079233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5201-ccc8-4a66-b54f-66a581d690fd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.080224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5201-ccc8-4a66-b54f-66a581d690fd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.080713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5201-ccc8-4a66-b54f-66a581d690fd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.082571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 347/713] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:54.088320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c9a8-5db3-484f-b04d-a293e86ecc02 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.089126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c9a8-5db3-484f-b04d-a293e86ecc02 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.089628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c9a8-5db3-484f-b04d-a293e86ecc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.109194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-fa92c9a8-5db3-484f-b04d-a293e86ecc02 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:54.115208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fa92c9a8-5db3-484f-b04d-a293e86ecc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:54.182764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 367/714] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:54.305212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e709-c02e-472d-9665-b002c1dbc660 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.305570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e709-c02e-472d-9665-b002c1dbc660 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.305823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3e709-c02e-472d-9665-b002c1dbc660 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.307337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 348/715] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:54.312695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.312987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.313266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.332081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:54.333267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:54.404194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 368/716] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:54.458160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3e709-c02e-472d-9665-b002c1dbc660 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:54.497954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:54.569444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3e709-c02e-472d-9665-b002c1dbc660 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:54.606180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:54.606852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.607134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.607412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.608025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:54.609388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:54.619088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:54.620128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.621000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 349/717] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Oct 19 13:17:54 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:54.628350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:54.675714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:54.713477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:54.745029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163234a-5c50-4ad1-8ff6-47c0ef2eb3b3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:54.793112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:54.795184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.795683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.796636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.798244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] RBAC: Adding query filter params (name=image) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:54.798617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:54.800003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:54.810619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 369/718] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:54.813231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:54.836974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:54.840761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5e8d9-253e-4d15-865a-4116af502ebb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:54.883388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:54.884068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.884361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.884897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.885490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:54.887089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:54.898630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:54.899767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 13:17:54.900639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:54.902588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 350/719] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:54.905639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:54.947349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573820c-2ff5-490d-8b66-535407f98b24 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:54.957812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:54.987391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:54.988009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:54.988294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:54.988574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:54.989276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] RBAC: Adding query filter params (type=image) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:54.989584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:54.991545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.002894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 370/720] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:17:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:55.040119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b24e-1e70-4d54-9400-d42224eb255a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:55.082950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:55.084746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.085274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.085891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.086828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:55.087948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:55.089594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6878a-a8d0-4f40-a738-80ae23838352 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.095958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 351/721] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:55.102204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:55.180571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0761-aa3d-4c98-bdd7-d43c7bcc1384 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.180953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0761-aa3d-4c98-bdd7-d43c7bcc1384 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.181180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0761-aa3d-4c98-bdd7-d43c7bcc1384 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.182756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 352/722] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:17:55.188604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.189201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.189678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.212941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:55.214283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:55.232604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a807b23-1a79-41a7-89b4-bc2e9633491e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:55.242752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:55.269995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:55.270644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.270928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.271205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.271841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:55.272146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:55.273517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a30555-26b0-4639-ab94-61a5fe7a7b4a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.284736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 371/723] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Oct 19 13:17:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:55.286245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cbbc4-fcca-44d5-8548-bb2932108382 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.286550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cbbc4-fcca-44d5-8548-bb2932108382 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.286829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cbbc4-fcca-44d5-8548-bb2932108382 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.292852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 353/724] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:55.292852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:55.293045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 372/725] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:55.294441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.294441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.294738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.312762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:55.313968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:55.357329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:55.405878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 373/726] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:17:55.514278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ed3bc-21ed-413a-8397-74de1920f3f3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:55.555521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:55.556163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.556453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.556731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.557401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:55.558804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.578850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:55.646624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 354/727] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:17:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:55.702348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc65f2-3ecf-4ea2-937d-fa0a7ea3de3a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:55.705073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:55.723209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:55.749053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:55.749675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.750213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.750517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.751175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:55.752599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.823859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 374/728] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:55 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:17:55.825077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35942f1-0157-4529-9ee1-4de18485caea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:55.883470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:55.884684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:55.884961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:55.885227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:55.885824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:55.887280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:55.898437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:55.899613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:55.900462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 355/729] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:55.906123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:55.935777 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:56.028938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771047d5-6d6b-4c40-a215-a0ebb77165d1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:56.064447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:56.065072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:56.065355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:56.065661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:56.066274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:56.066611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:56.068017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:56.073564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 375/730] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:56.079073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:56.164995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:56.192073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b502-9989-4068-be7b-4e9097a60f29 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:56.228584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:56.229502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:56.229966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:56.230422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:56.231289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:56.231775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:56.233433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ece49-0ff5-4bf2-8c2b-d9fb6ecc59f7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:56.239109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 356/731] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:56.247220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:56.339818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:56.355472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030540eb-e058-4182-9a75-138da6adb4cc None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:56.390702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:56.391307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:56.391590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:56.391863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:56.392524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:56.393884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff3440-199d-4db1-8bbe-c68213fa5f3a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:56.485234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 376/732] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:17:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:57.272631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57a5-1c5b-4ca3-9445-3d5ebe4193a6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:57.273140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57a5-1c5b-4ca3-9445-3d5ebe4193a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:57.273140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57a5-1c5b-4ca3-9445-3d5ebe4193a6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:57.274844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 357/733] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:57.280301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05dc3-b0ea-46db-8c9f-f49d518ed146 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:57.280699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05dc3-b0ea-46db-8c9f-f49d518ed146 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:57.280889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e05dc3-b0ea-46db-8c9f-f49d518ed146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:57.299304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-13e05dc3-b0ea-46db-8c9f-f49d518ed146 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:57.300472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-13e05dc3-b0ea-46db-8c9f-f49d518ed146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:57.380152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 377/734] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:57.643708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa57a5-1c5b-4ca3-9445-3d5ebe4193a6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:57.752431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa57a5-1c5b-4ca3-9445-3d5ebe4193a6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:57.787344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:57.788321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:57.788812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:57.789287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:57.790733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:57.792319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c7db2-95f1-4715-8451-6db3d4662992 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:57.872507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:57.880530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 358/735] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:17:57 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:17:57.982805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d342ec-bf0f-47b5-a77f-faf0cb6ca5d8 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:57.983118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d342ec-bf0f-47b5-a77f-faf0cb6ca5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:57.983341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d342ec-bf0f-47b5-a77f-faf0cb6ca5d8 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:57.985171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 378/736] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:57.991230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7d28e-7c92-4574-8b4c-28fcbc7891db None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:57.991555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7d28e-7c92-4574-8b4c-28fcbc7891db None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:57.991795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7d28e-7c92-4574-8b4c-28fcbc7891db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.012079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-58b7d28e-7c92-4574-8b4c-28fcbc7891db None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:58.013298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58b7d28e-7c92-4574-8b4c-28fcbc7891db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:58.093926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 359/737] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:58.116140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec058-d3d2-42f2-aa49-802e0cc0d676 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.116465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec058-d3d2-42f2-aa49-802e0cc0d676 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.116702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ec058-d3d2-42f2-aa49-802e0cc0d676 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.118353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 379/738] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:58.123635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.123917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.124186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.143544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:58.144713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:58.215578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 360/739] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:58 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:58.377974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ec058-d3d2-42f2-aa49-802e0cc0d676 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.450043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:58.485746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ec058-d3d2-42f2-aa49-802e0cc0d676 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:58.490070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.521803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:58.522417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.522724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.522994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.523650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:58.525694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:58.528320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING keystone.server.flask.common [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 19 13:17:58.559198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:58.589042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.sql.core [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 13:17:58.589042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:17:58.589042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [parameters: {'id': '10aada527af84a07b4c3037a33385a69', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 19 13:17:58.589042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81538) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 10aada527af84a07b4c3037a33385a69.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 10aada527af84a07b4c3037a33385a69. Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:17:58.611496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application result.read() Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 13:17:58.612415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application yield resource Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise self.value Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application session.commit() Oct 19 13:17:58.613432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self.session.flush() Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application compat.raise_( Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 13:17:58.614349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 13:17:58.615315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application result.read() Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:17:58.618016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [parameters: {'id': '10aada527af84a07b4c3037a33385a69', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.620509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:58.622522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 10aada527af84a07b4c3037a33385a69. Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 380/740] 10.222.0.39 () {60 vars in 1140 bytes} [Wed Oct 19 13:17:58 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:58.624558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.625753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce0bb-12e4-4151-a10e-fca40672f757 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:58.668623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:58.669849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.670365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.670914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.671833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:58.673509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8ec444a3-2ab1-4e51-a382-90c108654a2f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:58.685267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:58.686439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:17:58.687744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:17:58.690492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 361/741] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:58.692250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7319-c99e-4c40-ab24-1314774341ee None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.692855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7319-c99e-4c40-ab24-1314774341ee None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.693835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff7319-c99e-4c40-ab24-1314774341ee None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.695899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 362/742] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:17:58.697424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff7319-c99e-4c40-ab24-1314774341ee None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.750487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed797-73cc-45b6-b204-ccb0919f519e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:58.777910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:58.790520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:58.791165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.791511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.791860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.800576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:58.802238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:58.806586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff7319-c99e-4c40-ab24-1314774341ee None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-5e73a407-f1ac-4c98-ba23-58f61b28e2f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:17:58.814370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:58.815455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:58.816380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 381/743] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:17:58.817507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.817507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.817687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.831140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:58.837288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:17:58.838955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:17:58.849010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:58.849625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:58.849901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:58.850193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:58.850891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:58.851182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:58.852602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:58.857954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 363/744] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:58.858844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.923126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 382/745] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:17:58 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:17:58.924377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:58.971780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09abb-b194-4455-b74d-2e6c297cf148 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.017913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.018706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.019087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.019466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.020232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.021154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.022712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.046631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 364/746] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.047547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e852-57f6-4c78-98e8-cd8d60de2328 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.067298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:59.092561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.093202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.093471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.093746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.094395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] RBAC: Adding query filter params (type=compute) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.094792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.096200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.105728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 383/747] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:17:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.112302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.166454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:59.228714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.234663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea13848-d4ea-48c8-ac5d-9b17cff40a0f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.274193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.275977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.275977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.276173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.276892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] RBAC: Adding query filter params (interface=public, service_id=f99a6df915a346238e7e3cbb6aabdf99, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.277265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.279289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.284636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 365/748] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/endpoints?service_id=f99a6df915a346238e7e3cbb6aabdf99&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.290382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.358980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b5768-47ba-4781-9b76-0b683b51bfba None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.407986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29616f-9ec0-46ae-b348-a947b9a63f82 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.411786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.415346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.415787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.416212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.416964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.417835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:59.419852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:17:59.432885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:17:59.434217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:17:59.435289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:17:59.437985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 384/749] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:17:59.445733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.453671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.454357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.454736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.455073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.455747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.457177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.465924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 366/750] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:17:59.504342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:59.570077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f505de-268e-41f2-8ab0-443f09a5d0d0 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.608455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.609254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.609692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.609969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.610695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.611003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.612462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.618835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 385/751] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.627710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.736134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153176e-0409-4c35-a202-2bb95afb673b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.770415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.770971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.771250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.771521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.772144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.772441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.773801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.779756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 367/752] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.787693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:17:59.895132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4bc3a-c27d-409a-87f7-f43dc647a30b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:17:59.903345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:17:59.931768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:17:59.932378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:17:59.932661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:17:59.933001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:17:59.933716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] RBAC: Adding query filter params (interface=admin, service_id=88e1e891d4c44597b9319fd04495c0bf, region_id=RegionOne) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:17:59.934079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:17:59.935593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cddba4-13d9-4dc4-a1c6-e7e927eb2f22 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:17:59.941126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 386/753] 10.222.0.39 () {56 vars in 1249 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/endpoints?service_id=88e1e891d4c44597b9319fd04495c0bf&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:17:59.947553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:00.018128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:00.075782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a414bb-727b-4cfe-b928-2fcd44f870e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:00.114125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:00.114821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:00.115104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:00.115375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:00.115981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:00.117333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398507b8-88ac-4e87-9118-59db93b5d35d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:00.124871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 368/754] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:17:59 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:18:00.159626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eb2c2-301a-4cd1-82af-00443b2a679b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:00.159978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eb2c2-301a-4cd1-82af-00443b2a679b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:00.160190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eb2c2-301a-4cd1-82af-00443b2a679b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:00.161885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 387/755] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:00.167717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:00.168013 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:00.168288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:00.187488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:00.188751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:00.262589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 369/756] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:00 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:00.533353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eb2c2-301a-4cd1-82af-00443b2a679b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:00.640164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eb2c2-301a-4cd1-82af-00443b2a679b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:00.675352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:00.675993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:00.676339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:00.676622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:00.677214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:00.678685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:00.688366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:00.689368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:00.690218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 388/757] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Oct 19 13:18:00 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:00.695748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:00.723608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:00.801331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eddcd-d467-4000-bcb8-afd4d70734b8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:00.838260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:00.838899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:00.839175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:00.839452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:00.840079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:00.840379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:00.841739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:00.847131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 370/758] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:00.853137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:00.892196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:00.963133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7e468-0743-4229-8aee-f22f9ad612c4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:01.000952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:01.001736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.002125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.002501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.003262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] RBAC: Adding query filter params (type=image) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:01.003667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:01.005141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:01.010976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 389/759] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:01.016608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:01.117412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:01.127056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485286a-ef2a-48ad-8e3c-eb24ade628db None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:01.166593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:01.167366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.167741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.168113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.168910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] RBAC: Adding query filter params (interface=public, service_id=3340acee57024211b18539d99386835c, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:01.169319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:01.170797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7433e-8428-45e4-8c47-f13d305a18e8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:01.176531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 371/760] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:18:01 2022] GET /identity/v3/endpoints?service_id=3340acee57024211b18539d99386835c&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:01.183705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:01.288865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:01.293925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9af8-1a10-4569-862b-3fd5da6ae19d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:01.332060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:01.332852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.333303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.333931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.334704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:01.336222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9ce1-1d38-4e7e-93f6-af0d70030e84 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:01.344771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 390/761] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:18:01 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:18:01.432765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0654-b43b-4f54-895d-cb2307b0cd20 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.433119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0654-b43b-4f54-895d-cb2307b0cd20 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.433499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0654-b43b-4f54-895d-cb2307b0cd20 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.435020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 372/762] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:01.441084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9149-a78a-4f89-a924-37b094844449 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.441516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9149-a78a-4f89-a924-37b094844449 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.441894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9149-a78a-4f89-a924-37b094844449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.463497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2acb9149-a78a-4f89-a924-37b094844449 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:01.464880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2acb9149-a78a-4f89-a924-37b094844449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:01.542256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 391/763] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:01 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:01.767440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb133-de7a-4872-8c62-55f55fbf5130 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.768194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb133-de7a-4872-8c62-55f55fbf5130 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.768194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb133-de7a-4872-8c62-55f55fbf5130 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.769469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 373/764] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:01.774711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.775001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.775279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.793662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:01.794886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:01.822163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bb133-de7a-4872-8c62-55f55fbf5130 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:01.868595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 392/765] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:01 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:01.937778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bb133-de7a-4872-8c62-55f55fbf5130 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:01.973697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:01.974305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:01.974617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:01.974893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:01.976357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:01.978547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:01.982535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING keystone.server.flask.common [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 19 13:18:02.037028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 13:18:02.037028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 19 13:18:02.037028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [parameters: {'user_id': 'cc31bf6aa2fc4205857d7686f100d79c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 19 13:18:02.037028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:18:02.044335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:18:02.052379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 13:18:02.053353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 13:18:02.054284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:18:02.056397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:18:02.057240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cc31bf6aa2fc4205857d7686f100d79c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:02.058083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 19 13:18:02.059998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.062211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 374/766] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Oct 19 13:18:01 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:02.063189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.081717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.161103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.162426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993c7ea-4b10-43c5-b838-cb621948a997 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.184231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.199466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.200305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.200686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.201022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.209066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.210474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:18:02.222847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:02.224098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 13:18:02.225326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:02.231888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 393/767] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:02.233135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.296133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ed5a5-2168-45d4-be4b-d30f7de7a53a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.344367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.345148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.345535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.345907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.346736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.347240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.348915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:02.360916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:02.362788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:02.367309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 375/768] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:02.370830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.375935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9855d-7256-4938-8684-56b4bb44b022 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.411085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.412033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.412539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.413013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.413987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:02.415403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.417051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900097-731a-4f0e-994b-fe646f3c104f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.452040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 394/769] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:02.453654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff096-fd32-4f46-ac93-c1009a0f55fd None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.454141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff096-fd32-4f46-ac93-c1009a0f55fd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.454657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff096-fd32-4f46-ac93-c1009a0f55fd None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.456119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 395/770] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:02.462775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.463064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.463328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.470468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.485208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:02.486835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:02.495564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c18c5-77db-4358-8572-d26e820af986 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.538656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.539286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.539570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.540411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.541059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:02.541391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.542798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.554763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 376/771] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:02.558890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.576622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 396/772] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:02 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:02.675957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2f02b-650e-43fb-bfdc-ffab6266ffa3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.711473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.712681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.713039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.713386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.714077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:02.714446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.715884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.721306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 377/773] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:02.728227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.801238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.831051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f52ea-a217-4230-92fc-6b7bc46fb76d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.849416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:02.862463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:02.870148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.874821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.874821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.874821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.874821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.874821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:02.951106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af4dfe-b817-490b-aa21-478bde74e5ce None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:02.962716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 397/774] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:18:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:02.996048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:02.996703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:02.996987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:02.997261 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:02.997848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:02.999299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:03.020933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:03.022024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:03.023554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 378/775] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Oct 19 13:18:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:03.028910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:03.146588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c367-fc16-4b95-a1cc-706d7315ff4d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:03.173726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:03.182379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:03.183029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:03.183309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:03.184734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:03.184734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:03.184734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:03.186140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:03.191412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 398/776] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:18:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:03.197492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:03.277274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:03.308060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8f6d-db3e-410b-bd56-0fee38fbbf77 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:03.342986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:03.343595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:03.343878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:03.344155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:03.344782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:03.345085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:03.346453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b1ae-06ce-4b56-b7df-ebff8924d424 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:03.356076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 379/777] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Oct 19 13:18:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:03.357355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:03.466475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d70a0d-a287-4887-bbe3-d5b082ade06d None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:03.500914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:03.501534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:03.501820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:03.502098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:03.502804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:03.504185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:03.534365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b4ec-1726-4bc3-8b59-ecd73349e26b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:03.534688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b4ec-1726-4bc3-8b59-ecd73349e26b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:03.534971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b4ec-1726-4bc3-8b59-ecd73349e26b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:03.536308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 380/778] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:18:03.541571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:03.541867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:03.542148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:03.561640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:03.563198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:03.612282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 399/779] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:18:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:03.641334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 381/780] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:03.907377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:03.963309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:04.020166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca610a-3dcb-4bf8-9e9c-1302f280a2fa None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:04.055066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:04.055732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:04.056017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:04.056292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] PATH_INFO: `/v3/services/image` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:04.056871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:04.058310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:04.067252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:04.068232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:04.071396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 400/781] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Oct 19 13:18:03 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:04.074836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:04.138035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:04.179304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b904d-b627-419c-ad5a-5f8576f0640e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:04.212696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:04.213347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:04.213634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:04.213905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:04.214595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:04.214897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:04.216245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:04.225719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 382/782] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:04.227778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:04.330845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb0bc0-b24a-40c4-a9bb-e50b376124cd None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:04.345739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:04.366399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196132e-73a8-429d-b534-628ea385099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:04.367110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196132e-73a8-429d-b534-628ea385099d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:04.367396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196132e-73a8-429d-b534-628ea385099d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:04.367675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196132e-73a8-429d-b534-628ea385099d None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:04.368335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196132e-73a8-429d-b534-628ea385099d None admin] RBAC: Adding query filter params (type=image) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:04.368641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196132e-73a8-429d-b534-628ea385099d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:04.370026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196132e-73a8-429d-b534-628ea385099d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:04.379550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 401/783] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:04.381269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:04.481997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ed18-149d-4bed-8b47-0a5b1c13acf6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:04.506330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:04.517860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:04.518471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:04.518776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:04.519047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:04.519766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:04.521128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b35d-67bd-4f00-b3ad-48384978b1bb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:04.603080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 383/784] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:18:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:18:05.203224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11495c37-daf1-461c-b869-d4ec351ef9b1 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.203770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11495c37-daf1-461c-b869-d4ec351ef9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.203770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11495c37-daf1-461c-b869-d4ec351ef9b1 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.205239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 402/785] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:05.210454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.210852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.211026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.229214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:05.230376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:05.306671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 384/786] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:05.570355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11495c37-daf1-461c-b869-d4ec351ef9b1 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:05.677802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11495c37-daf1-461c-b869-d4ec351ef9b1 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:05.714586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:05.715217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.715495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.715767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.727110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-cfcf62f6-076d-4bee-8bd7-77a7fdd5f2b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:05.732388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:05.733299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:05.738921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 403/787] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:18:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:05.740150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:05.776096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76763-11ef-47bc-b2ee-853d4b6dae5f None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.776394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76763-11ef-47bc-b2ee-853d4b6dae5f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.776654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe76763-11ef-47bc-b2ee-853d4b6dae5f None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.778120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 404/788] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:18:05.782835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a814c-bd55-4002-84d9-d01156953982 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.783123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a814c-bd55-4002-84d9-d01156953982 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.783405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a814c-bd55-4002-84d9-d01156953982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.802281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-033a814c-bd55-4002-84d9-d01156953982 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:05.803507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-033a814c-bd55-4002-84d9-d01156953982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:05.840155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:05.846381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c44c2-6ea1-4613-9e64-052839ad8463 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:05.855026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:05.881618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:05.882580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:05.883050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:05.883826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:05.884517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:05.884818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:05.886244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:05.886645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 405/789] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:05.894876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 385/790] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Oct 19 13:18:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:05.901044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a814c-bd55-4002-84d9-d01156953982 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.007138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a814c-bd55-4002-84d9-d01156953982 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.042558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.043244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.043522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.043794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.053806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.055368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:18:06.069835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:06.071516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:18:06.072473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:18:06.072473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:06.072473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:06.073038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 406/791] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:06.077421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.147694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.173830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:06.182470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f9ee7-b4e2-4f24-ab59-2d3dbb929532 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.218860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.219477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.219733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.220007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.220794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:06.221131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.222837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.234739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 386/792] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:06.240377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.264154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be8b6f-01de-441e-bee6-157117b4770b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.270462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:06.300988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.301879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.302162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.302431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.303539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.304957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.307082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 407/793] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:06.314005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.349703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede282f-a902-4c04-81a8-f2acae6cfbd0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.389208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.389842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.390128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.390403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.398530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.400316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.411147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:18:06.412507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:06.413456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:06.415175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 387/794] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:06.419292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.425724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5618d47-026e-43a8-a9dc-8cc0b5efc1ba None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.462269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.462952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.463234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.463501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.464533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.466404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.515154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:06.526702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f9428-b24a-47b6-85cb-7998b0f64c96 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.558555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 408/795] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:18:06 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:06.561449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.562084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.562364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.562673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.563425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:06.563734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.565420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa000-23c5-4736-8bc5-76d213be0c12 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.592627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 388/796] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:06.594090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.709043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234175e5-4284-4eb3-9f8c-7ac6bf482b4b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.748692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.749335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.749613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.749882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.758278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:06.761580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.763105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.769504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4e3d-3274-44c8-b5a4-8c5de8c6a0c1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.770613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4e3d-3274-44c8-b5a4-8c5de8c6a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.770613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4e3d-3274-44c8-b5a4-8c5de8c6a0c1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.771400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 389/797] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:18:06.776410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.776698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.776974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:18:06.778639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:06.779749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:18:06.780655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:18:06.780655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:06.780655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:06.780655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 409/798] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:06.785758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:06.798827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:06.800268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:06.872305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 390/799] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:06.899806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ac061-6dc4-4382-b6b6-997bc3d96cba None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:06.936378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:06.936990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:06.937274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:06.937553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:06.938197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:06.938514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:06.939891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:06.952142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 410/800] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:06.957740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.044049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:07.067219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78a15-ba73-4f6b-ba19-b6b827c9235a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.103478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.104111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.104437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.104717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.111472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.112912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:18:07.125113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:07.126025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:18:07.127338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.130814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 391/801] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:07.132152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.139955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.197540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:07.246395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7315-d4df-4472-b421-2b58999a7557 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.247417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed14593-d8a7-4907-a018-0d979d44b050 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.280559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.281091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.281387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.281656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.282322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.283731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.285109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.286247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.286755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.287247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.288227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:07.288644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:07.289317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.291258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.303231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 411/802] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:07.309052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.334456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:18:07.334456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:18:07.334456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [parameters: {'id': '4870d2275caa41ca954cea3ed61eca91', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 13:18:07.334456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:18:07.345279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 13:18:07.346105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 13:18:07.347569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 13:18:07.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 13:18:07.350161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 13:18:07.356140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '4870d2275caa41ca954cea3ed61eca91', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 13:18:07.359057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:07.361011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 392/803] 10.222.0.39 () {60 vars in 1137 bytes} [Wed Oct 19 13:18:07 2022] POST /identity/v3/domains => generated 190 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:07.363167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.415922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ca7b-50ae-4c84-ad84-d09709dc1798 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.429249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:07.450906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.451587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.451854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.452131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/f81416c420b3420c85b79049156eaf8f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.454533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=f025b86069754817868da5d0b19e3de0, role_id=f81416c420b3420c85b79049156eaf8f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.455979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.472562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51290b5d-edb9-4fa7-819b-63f3450843ed None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.509326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.510069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.510345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.510651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.521518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.523072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.529369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 412/804] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:18:07 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/f025b86069754817868da5d0b19e3de0/roles/f81416c420b3420c85b79049156eaf8f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3cfd4e83-db4a-479f-8e2f-f6d071b6744d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:18:07.538203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:07.539310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:18:07.550131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:18:07.550131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:07.550131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:07.550131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 393/805] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:07.550571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:07.660563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1e4fa-95e5-4be9-8534-b91832d1a665 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:07.696349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:07.697119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:07.697401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:07.697672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:07.698313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:07.698655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:07.700034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e798390-4754-4ec9-9c4d-41df66aba80b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:07.713732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 413/806] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:08.756799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5298-35ab-462c-9080-2f5fa3633740 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:08.757326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5298-35ab-462c-9080-2f5fa3633740 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:08.757326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c5298-35ab-462c-9080-2f5fa3633740 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:08.759365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 394/807] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:08.764514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:08.764899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:08.765064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:08.783625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:08.784799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:08.809861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:08.850180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 414/808] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:09.123700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c5298-35ab-462c-9080-2f5fa3633740 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:09.175980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:09.223132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c5298-35ab-462c-9080-2f5fa3633740 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:09.255563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:09.256188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:09.256467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:09.256738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:09.257836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:09.259317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1404fdd-8a47-45db-896a-7d072b72d82f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:09.260866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 395/809] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:09.266552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:09.362748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500e42-4e2a-4198-8ab5-1c172543c22e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:09.394771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:09.395307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:09.395584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:09.395853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:09.396499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:09.397912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa90b4-acc8-4979-85a9-ec42e7caf7f0 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:09.497109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 415/810] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:18:09 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:09.683772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cf9c-a6ba-4f49-b03d-e1b55564af64 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:09.683965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cf9c-a6ba-4f49-b03d-e1b55564af64 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:09.684253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cf9c-a6ba-4f49-b03d-e1b55564af64 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:09.685743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 396/811] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:09.691018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:09.691313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:09.691591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:09.710713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:09.712465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:09.725829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:09.778964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 416/812] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:10.042095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9cf9c-a6ba-4f49-b03d-e1b55564af64 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.141559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9cf9c-a6ba-4f49-b03d-e1b55564af64 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.161219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:10.174645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.175276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.175557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.175944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.186412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:10.191313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:10.192330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:10.193843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 397/813] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:10.198933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.296733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361540da-006f-47ff-8cc2-51da45ec9686 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.330847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.331452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.331735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.332010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.332678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:10.332982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:10.334624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:10.342700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 417/814] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:10.348343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.450694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e52ad-e6e1-45ef-9c33-e27b6f73ed03 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.483787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb138-c576-406e-a514-484851830c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.484407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb138-c576-406e-a514-484851830c27 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.484691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb138-c576-406e-a514-484851830c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.484966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb138-c576-406e-a514-484851830c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.495435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb138-c576-406e-a514-484851830c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:10.496861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb138-c576-406e-a514-484851830c27 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-338bb138-c576-406e-a514-484851830c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:18:10.510889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:10.511948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:18:10.512863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:18:10.512863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:10.512863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:10.512863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 398/815] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:10.517820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.530244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:10.614626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345194-d958-4b6c-b226-9cd28b0d0b28 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.646715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.647315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.647593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.647861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.648488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:10.648810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:10.650158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:10.661422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 418/816] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:10.667905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb138-c576-406e-a514-484851830c27 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.699136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:10.769900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb138-c576-406e-a514-484851830c27 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.804523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.805184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.805466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.805740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.812660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:10.814085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:18:10.825440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:10.826549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:10.827929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 399/817] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:10.832234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:10.898302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:10.927555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841f5bc-dc80-496c-9d48-536b0832b6fa None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:10.963367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:10.963970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:10.964252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:10.964524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:10.965247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:10.965579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:10.967030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:10.984322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 419/818] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:10.990055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.082091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:11.091593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21c94f-64e9-4566-bd95-053e3fdd9771 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:11.124915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:11.125512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.125787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.126053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.190995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:11.192357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:11.278209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:11.287168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 400/819] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:18:11.293071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.388522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0915478-aadc-4171-b166-389d2c64e8d4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:11.414041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:11.421491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:11.427453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.427453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.427453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.432118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:11.433547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:18:11.448380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:11.449489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:18:11.450722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:18:11.450722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:18:11.450722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:11.454184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 420/820] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:11.455056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.552314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2fbfa-3585-4c94-a432-7b9875baf904 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:11.589258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:11.589863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.590142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.590415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.591084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:11.591386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:11.592747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93701a-0c43-4168-84e3-0d2675125c71 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:11.603787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 401/821] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:11.609424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.654066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cc2cb-d458-4fbd-a472-a593fe1783cd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.654535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cc2cb-d458-4fbd-a472-a593fe1783cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.654822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cc2cb-d458-4fbd-a472-a593fe1783cd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.656153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 402/822] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:18:11.660957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.661250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.661525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.679171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:11.680924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:11.707316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1515f62-abe6-42ab-a42d-9e7ca2178001 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:11.727901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:11.740983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:11.744372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.744372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.744372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.748123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 403/823] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:11.749597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:11.751103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:18:11.763326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:11.764314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:11.765227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 421/824] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:11.770059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.865287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5207d-7b39-4590-b209-2aa42bd64516 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:11.898399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:11.899021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:11.899304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:11.899575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:11.900259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:11.900562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:11.901981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:11.913090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 404/825] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:11.919438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:11.960577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:12.011842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:12.021508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a0d9c-12b8-4f75-8438-954d68d75dfe None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.054912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.063135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.063135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.063135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.063135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] RBAC: Adding query filter params (role.id=f81416c420b3420c85b79049156eaf8f, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=f025b86069754817868da5d0b19e3de0) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:12.063135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:12.063567 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:12.063658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:12.065171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 422/826] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:18:11 2022] GET /identity/v3/role_assignments?user.id=f025b86069754817868da5d0b19e3de0&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=f81416c420b3420c85b79049156eaf8f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:12.120926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9e423-50ac-4b90-93c2-f6593ea3e661 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.156752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.157393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.157668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.157938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.169373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:12.174789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:12.175931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:12.176815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 405/827] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:18:12.182804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:12.287465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819be0-a72b-44e8-b911-f7b3d225eb68 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.290287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:12.321458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.322307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.322642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.323076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.324352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:12.324352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:12.325721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:12.335479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 423/828] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:12.341814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:12.448375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc9ec4-b115-45c1-a1ce-5d6b9c74cb7a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.449920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:12.485201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.485837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.486117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.486391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] PATH_INFO: `/v3/users/95d81bdc61954e7ca3d0a3c1f94fc878` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.499941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] RBAC: Authorizing `identity:get_user(user_id=95d81bdc61954e7ca3d0a3c1f94fc878)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:12.501374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:12.503216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 406/829] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:12 2022] GET /identity/v3/users/95d81bdc61954e7ca3d0a3c1f94fc878 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:12.508853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:12.608638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb0448-2446-4e39-9a33-1e7043ac810a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.643722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.644362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.644646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.644925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.646036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f0c0f50740854aeca528c1611b5913f2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:12.647629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbec2c5-6833-44f1-a95a-537a65c903f7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:12.649900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 424/830] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:12 2022] GET /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:12.655953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:12.757656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bee2a-fdfa-4736-a042-caedde50dd43 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:12.793937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:12.794619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:12.794903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:12.795178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2/users/95d81bdc61954e7ca3d0a3c1f94fc878/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:12.797051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0c0f50740854aeca528c1611b5913f2, user_id=95d81bdc61954e7ca3d0a3c1f94fc878, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:12.798470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53815-a3d1-40dd-a2c3-573b70e57be7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:12.911325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 407/831] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:18:12 2022] PUT /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2/users/95d81bdc61954e7ca3d0a3c1f94fc878/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:18:14.986933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4abb-c8bb-4210-908f-61ca91941e40 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:14.987490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4abb-c8bb-4210-908f-61ca91941e40 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:14.987490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4abb-c8bb-4210-908f-61ca91941e40 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:14.988818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 425/832] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:14.994419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:14.994880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:14.995016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:15.014623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:15.015802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:15.029491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:15.079257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 408/833] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:15.349266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb4abb-c8bb-4210-908f-61ca91941e40 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:15.383839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:15.449058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb4abb-c8bb-4210-908f-61ca91941e40 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:15.481621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:15.482255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:15.482569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:15.482851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:15.493333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:15.498122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:15.502592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:15.503693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 426/834] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:15.505204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:15.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d6ed7-1978-404c-adfc-ecc263770de2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:15.632572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:15.633163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:15.633441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:15.633713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:15.634366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:15.634696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:15.636050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:15.643733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 409/835] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:15.649450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:15.739557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:15.746651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d5d2f-9407-47a6-bd99-13d5739489ce None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:15.778820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:15.779432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:15.779716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:15.779999 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] PATH_INFO: `/v3/users/95d81bdc61954e7ca3d0a3c1f94fc878` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:15.781114 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] RBAC: Authorizing `identity:get_user(user_id=95d81bdc61954e7ca3d0a3c1f94fc878)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:15.782564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:15.784103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 427/836] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:15 2022] GET /identity/v3/users/95d81bdc61954e7ca3d0a3c1f94fc878 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:15.790271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:15.805504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:15.885963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ca3c-a4b7-40ac-aa34-6c026b8d8031 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:15.918011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:15.918619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:15.918897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:15.919164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:15.984043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:15.985499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:15.998211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:16.080506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 410/837] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:18:16.087111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:16.183167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daa67-d5bf-4d38-94cb-491dfd250099 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:16.216071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:16.216684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:16.216963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:16.217235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:16.218324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] RBAC: Authorizing `identity:get_project(project_id=f0c0f50740854aeca528c1611b5913f2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:16.219775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68554324-4c69-426b-98f2-c840077bdd17 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:16.221352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 428/838] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:16 2022] GET /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:16.227791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:16.320789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b3069-7302-4da7-a351-5870ec176e44 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:16.332771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:16.353625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:16.354748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:16.354748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:16.354748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:16.355452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=f0c0f50740854aeca528c1611b5913f2, user.id=95d81bdc61954e7ca3d0a3c1f94fc878) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:16.355753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:16.357145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7720a8-b7c9-4525-a2a3-836487f92eac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:16.362617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 411/839] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:18:16 2022] GET /identity/v3/role_assignments?user.id=95d81bdc61954e7ca3d0a3c1f94fc878&scope.project.id=f0c0f50740854aeca528c1611b5913f2&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:18.473517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264a241-879f-4a1d-a674-a529c1a0d409 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:18.474094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264a241-879f-4a1d-a674-a529c1a0d409 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:18.474094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264a241-879f-4a1d-a674-a529c1a0d409 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:18.475531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 429/840] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:18.480744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:18.481116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:18.481294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:18.500200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:18.501382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:18.566208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 412/841] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:18.833933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264a241-879f-4a1d-a674-a529c1a0d409 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:18.863743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:18.933507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264a241-879f-4a1d-a674-a529c1a0d409 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:18.966051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:18.966695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:18.966976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:18.967247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:18.968332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:18.969773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aebc62-a7ff-4afe-b5fa-d0e8bba5829c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:18.971338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 430/842] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:18.977698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:19.074380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a9d1b-9e06-4ddf-a933-42f9ad6d871e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:19.081555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:19.108224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:19.108764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:19.109105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:19.109379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:19.110027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:19.111479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545295b-f3c2-40f1-9b13-e5ccf8b0437d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:19.213649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 413/843] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:18:18 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:21.361752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123f5b7-569a-40ad-8852-3fefca748f6f None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:21.362273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123f5b7-569a-40ad-8852-3fefca748f6f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:21.362380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123f5b7-569a-40ad-8852-3fefca748f6f None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:21.363658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 431/844] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:21.369192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:21.369630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:21.369751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:21.388219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:21.389413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:21.453446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 414/845] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:21.719857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123f5b7-569a-40ad-8852-3fefca748f6f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:21.814561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:21.819356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123f5b7-569a-40ad-8852-3fefca748f6f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:21.852482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:21.853321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:21.853776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:21.854361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] PATH_INFO: `/v3/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:21.855806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] RBAC: Authorizing `identity:get_role(role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:21.857188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:21.858498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 432/846] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:21 2022] GET /identity/v3/roles/2aaa59501e4146f5adce6e3872912797 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:21.864241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:21.952463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:21.961574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e974-ad31-4df8-817e-ce5ffe764872 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:21.995299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:21.996205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:21.996749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:21.997397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] PATH_INFO: `/v3/users/ff6daeb8c03b433bae8b352d61512776` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:22.010715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6daeb8c03b433bae8b352d61512776)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:22.012157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:22.013800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 415/847] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:21 2022] GET /identity/v3/users/ff6daeb8c03b433bae8b352d61512776 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:22.019643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:22.117157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ab29-ebd8-4992-9bf1-8f14b1a3b1d9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:22.149446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:22.150057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:22.150335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:22.150640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:22.151730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] RBAC: Authorizing `identity:get_project(project_id=f0c0f50740854aeca528c1611b5913f2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:22.153145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ec70-1a35-40c2-905a-d78834d6972d None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:22.154766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 433/848] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:22 2022] GET /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:22.160652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:22.274245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4313cdd-5f12-4713-bd2a-a5af422f6423 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:22.306716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:22.307353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:22.307631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:22.307899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2/users/ff6daeb8c03b433bae8b352d61512776/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:22.309978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0c0f50740854aeca528c1611b5913f2, user_id=ff6daeb8c03b433bae8b352d61512776, role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:22.311419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2602-dca2-4348-8a38-d33011b1c8c8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:22.399686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 416/849] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:18:22 2022] PUT /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2/users/ff6daeb8c03b433bae8b352d61512776/roles/2aaa59501e4146f5adce6e3872912797 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:18:24.496028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997ea9f-acad-43aa-8a58-6aae1eaf1b0b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:24.496596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997ea9f-acad-43aa-8a58-6aae1eaf1b0b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:24.496596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997ea9f-acad-43aa-8a58-6aae1eaf1b0b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:24.497908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 434/850] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:24.503037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:24.503429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:24.503588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:24.521098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:24.522269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:24.525455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:24.588720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 417/851] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:24.858002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997ea9f-acad-43aa-8a58-6aae1eaf1b0b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:24.892332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:24.956437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997ea9f-acad-43aa-8a58-6aae1eaf1b0b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:24.990180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:24.990835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:24.991122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:24.991401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] PATH_INFO: `/v3/roles/2aaa59501e4146f5adce6e3872912797` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:24.992825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] RBAC: Authorizing `identity:get_role(role_id=2aaa59501e4146f5adce6e3872912797)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:24.994187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:24.995540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 435/852] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:24 2022] GET /identity/v3/roles/2aaa59501e4146f5adce6e3872912797 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:25.001194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:25.097971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899809b-9115-4d19-b1bc-be70af515153 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:25.131030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:25.131596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:25.131876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:25.132149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] PATH_INFO: `/v3/users/ff6daeb8c03b433bae8b352d61512776` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:25.133302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] RBAC: Authorizing `identity:get_user(user_id=ff6daeb8c03b433bae8b352d61512776)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:25.134761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:25.136314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 418/853] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:25 2022] GET /identity/v3/users/ff6daeb8c03b433bae8b352d61512776 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:25.142029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:25.238017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802902b8-2b08-4a3a-870d-9abf3f820cf1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:25.239537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:25.270677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:25.271266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:25.271543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:25.271814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:25.342080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:25.343571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:25.443986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 436/854] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:18:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:18:25.450085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:25.462395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:25.551265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2e7a6-7a2a-47b2-8c62-92e6378973e1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:25.583564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:25.584178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:25.584454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:25.584724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] PATH_INFO: `/v3/projects/f0c0f50740854aeca528c1611b5913f2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:25.586445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] RBAC: Authorizing `identity:get_project(project_id=f0c0f50740854aeca528c1611b5913f2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:25.587886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8fc9-d9b1-49a0-bde6-5a5d3c7e5afa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:25.589486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 419/855] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:25 2022] GET /identity/v3/projects/f0c0f50740854aeca528c1611b5913f2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:25.595785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:25.597989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:25.690873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea2ffc-7ffc-404f-a02d-87a3111e43d4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:25.724721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:25.725248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:25.725527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:25.725801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:25.726557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] RBAC: Adding query filter params (role.id=2aaa59501e4146f5adce6e3872912797, scope.project.id=f0c0f50740854aeca528c1611b5913f2, user.id=ff6daeb8c03b433bae8b352d61512776) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:25.726868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:25.728283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741dc4fa-9fea-42d1-8cb7-c3684a716eb9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:25.733843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 437/856] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:18:25 2022] GET /identity/v3/role_assignments?user.id=ff6daeb8c03b433bae8b352d61512776&scope.project.id=f0c0f50740854aeca528c1611b5913f2&role.id=2aaa59501e4146f5adce6e3872912797 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:27.838981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca33e-bdd5-414d-b5e3-3065d5c62a57 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:27.839513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca33e-bdd5-414d-b5e3-3065d5c62a57 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:27.839513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ca33e-bdd5-414d-b5e3-3065d5c62a57 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:27.840903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 420/857] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:27.846190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:27.846652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:27.846780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:27.979083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:27.980290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:28.039329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:28.046522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 438/858] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:28.314436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca33e-bdd5-414d-b5e3-3065d5c62a57 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:28.390620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:28.412912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ca33e-bdd5-414d-b5e3-3065d5c62a57 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:28.446383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:28.447025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:28.447309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:28.447583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:28.448689 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:28.450114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c61a54-af24-420a-ad44-77d52bb51ffc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:28.451687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 421/859] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:28.457266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:28.552418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d215c8-2ca7-4b10-973f-14eb1ef96398 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:28.585285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:28.585848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:28.586131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:28.586407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:28.587095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:28.588495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1e832-b0de-4a5f-84fb-beab9eb828d7 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:28.663078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 439/860] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:18:28 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:30.818778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dda34-c417-43e2-a457-16af9aa365a1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:30.819337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dda34-c417-43e2-a457-16af9aa365a1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:30.819337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dda34-c417-43e2-a457-16af9aa365a1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:30.821197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 422/861] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:30.826410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:30.826811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:30.826991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:30.845182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:30.846347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:30.888057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:30.910247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 440/862] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:31.176914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dda34-c417-43e2-a457-16af9aa365a1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:31.276033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dda34-c417-43e2-a457-16af9aa365a1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:31.308463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:31.309080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:31.309363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:31.309639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:31.310772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:31.312197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375f2ac-3b18-48ce-a021-b87e702b1fd9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:31.313738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 423/863] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:18:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:31.319274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:31.414136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99940bc-fe1f-418c-aad5-24771c0a58c4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:31.446697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:31.447235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:31.447514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:31.447786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:31.448435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:31.449852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94be99-307b-4e5a-b763-e9966c2493da None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:31.539320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 441/864] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:18:31 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:33.663337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12929d06-d06c-44c6-8358-54811f167e99 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:33.664380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12929d06-d06c-44c6-8358-54811f167e99 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:33.664380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12929d06-d06c-44c6-8358-54811f167e99 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:33.665720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 424/865] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:33.670857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:33.671214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:33.671411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:33.689511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:33.690712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:33.716245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:33.754568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 442/866] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:34.019482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12929d06-d06c-44c6-8358-54811f167e99 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:34.034620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:34.118516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12929d06-d06c-44c6-8358-54811f167e99 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:34.151596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:34.152217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:34.152498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:34.152770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:34.162590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:34.167960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:34.169070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:34.170178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 425/867] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:34.175104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:34.268422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ab0a7-0493-48d8-ac3e-a1388b27f3e2 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:34.302023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:34.302677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:34.302937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:34.303227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:34.303890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:34.304191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:34.305542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:34.313437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 443/868] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:34.319214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:34.388365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:34.416356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b255-b076-44af-a3fd-e793647a78e3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:34.448681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:34.449285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:34.449563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:34.449833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] PATH_INFO: `/v3/users/12d5eb40b9fc4152bbd90033cd51c187` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:34.463001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] RBAC: Authorizing `identity:get_user(user_id=12d5eb40b9fc4152bbd90033cd51c187)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:34.464427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:34.465975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 426/869] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:34 2022] GET /identity/v3/users/12d5eb40b9fc4152bbd90033cd51c187 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:34.471632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:34.498216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:34.568343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a8c5f-7731-42c3-827b-36b8e6d53415 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:34.601072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:34.601729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:34.602006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:34.602275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] PATH_INFO: `/v3/projects/49c1e1d6316c47488e229a9d634891dd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:34.603469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] RBAC: Authorizing `identity:get_project(project_id=49c1e1d6316c47488e229a9d634891dd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:34.604870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1e44-9a65-421c-a3a5-e1aa50624b62 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:34.606456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 444/870] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:34 2022] GET /identity/v3/projects/49c1e1d6316c47488e229a9d634891dd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:34.612215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:34.706492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41154666-c5ae-4639-a034-c685fa9e3add None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:34.733879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:34.739358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:34.739998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:34.740274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:34.740544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] PATH_INFO: `/v3/projects/49c1e1d6316c47488e229a9d634891dd/users/12d5eb40b9fc4152bbd90033cd51c187/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:34.742907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] RBAC: Authorizing `identity:create_grant(project_id=49c1e1d6316c47488e229a9d634891dd, user_id=12d5eb40b9fc4152bbd90033cd51c187, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:34.744308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e62c42-b1e5-463a-acba-2900a6dde34e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:34.829523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 427/871] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:18:34 2022] PUT /identity/v3/projects/49c1e1d6316c47488e229a9d634891dd/users/12d5eb40b9fc4152bbd90033cd51c187/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:18:36.911577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09ba5a-185d-43d2-b70c-b8fe297f566b None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:36.912100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09ba5a-185d-43d2-b70c-b8fe297f566b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:36.912100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09ba5a-185d-43d2-b70c-b8fe297f566b None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:36.913961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 445/872] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:36.919256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:36.919654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:36.919804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:36.938069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:36.939284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:37.003162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 428/873] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:18:37.273837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09ba5a-185d-43d2-b70c-b8fe297f566b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:37.365092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:37.372392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09ba5a-185d-43d2-b70c-b8fe297f566b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:37.406187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:37.406979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:37.407258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:37.407530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:37.417334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:37.422079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:37.423206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:37.424103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 446/874] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:37.429101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:37.525008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03e555-9f75-4092-8432-96b907a8cf0d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:37.543917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:37.557424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:37.558017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:37.558295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:37.558591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:37.559245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:37.559544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:37.560982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:37.568670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 429/875] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:37.574988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:37.671869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349aa1c-a7df-47a2-b132-4beaf08a9e73 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:37.704495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:37.705106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:37.705389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:37.705664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] PATH_INFO: `/v3/users/12d5eb40b9fc4152bbd90033cd51c187` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:37.706798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] RBAC: Authorizing `identity:get_user(user_id=12d5eb40b9fc4152bbd90033cd51c187)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:37.708495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:37.710040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 447/876] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:37 2022] GET /identity/v3/users/12d5eb40b9fc4152bbd90033cd51c187 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:37.716351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:37.810925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8e91-f4c9-44df-9c14-1a7bf0334d7f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:37.843102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:37.843849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:37.844129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:37.844402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:37.891383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:37.908670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:37.910086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:38.004047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 430/877] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:18:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 19 13:18:38.010426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:38.034241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:38.107768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c5873-a085-436b-8ad0-eb926c5c4d67 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:38.140394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:38.141009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:38.141291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:38.141568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] PATH_INFO: `/v3/projects/49c1e1d6316c47488e229a9d634891dd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:38.142698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] RBAC: Authorizing `identity:get_project(project_id=49c1e1d6316c47488e229a9d634891dd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:38.144156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56bb9-c18d-4cb1-8913-bca870e8d33e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:38.145792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 448/878] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:38 2022] GET /identity/v3/projects/49c1e1d6316c47488e229a9d634891dd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:38.151949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:38.229800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:38.247244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266bed8-80fb-461b-a525-a73e35a8fe00 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:38.279369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:38.279957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:38.280233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:38.280502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:38.281216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=49c1e1d6316c47488e229a9d634891dd, user.id=12d5eb40b9fc4152bbd90033cd51c187) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:38.281519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:38.282955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b89e29-2cb5-4a81-b15d-765c25f00be8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:38.288972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 431/879] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:18:38 2022] GET /identity/v3/role_assignments?user.id=12d5eb40b9fc4152bbd90033cd51c187&scope.project.id=49c1e1d6316c47488e229a9d634891dd&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:18:40.412352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59db60-df67-41c1-ac22-e0f3564e9452 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:40.412945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59db60-df67-41c1-ac22-e0f3564e9452 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:40.412945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59db60-df67-41c1-ac22-e0f3564e9452 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:40.414789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 449/880] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:40.419911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a688b-cb67-45f5-9a9a-bfe360a200e4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:40.420301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a688b-cb67-45f5-9a9a-bfe360a200e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:40.420463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a688b-cb67-45f5-9a9a-bfe360a200e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:40.438697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3d1a688b-cb67-45f5-9a9a-bfe360a200e4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:40.439866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3d1a688b-cb67-45f5-9a9a-bfe360a200e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:40.503782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 432/881] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:40.777856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59db60-df67-41c1-ac22-e0f3564e9452 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:40.866412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:40.876523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59db60-df67-41c1-ac22-e0f3564e9452 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:40.909926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:40.910537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:40.910819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:40.911089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:40.911743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:40.913085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7e13b-4e16-4bf1-8a83-c0b0217c1d0a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:40.984717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 450/882] 10.222.0.39 () {60 vars in 1138 bytes} [Wed Oct 19 13:18:40 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:43.106671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a0626-90bd-436a-ba97-4c1e7321477b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:43.107219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a0626-90bd-436a-ba97-4c1e7321477b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:43.107219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a0626-90bd-436a-ba97-4c1e7321477b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:43.108556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 433/883] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:43.113777 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:43.114208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:43.114310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:43.132344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:43.134065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:43.197201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 451/884] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:43.461867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a0626-90bd-436a-ba97-4c1e7321477b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:43.536865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:43.560274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a0626-90bd-436a-ba97-4c1e7321477b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:43.592866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:43.593480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:43.593764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:43.594038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] PATH_INFO: `/v3/domains/99757ed348b847b1b1ba818dcce75f3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:43.604690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=99757ed348b847b1b1ba818dcce75f3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:43.606119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5a74-2287-4b91-a5e0-9a3f0293b066 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:43.607713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 434/885] 10.222.0.39 () {56 vars in 1183 bytes} [Wed Oct 19 13:18:43 2022] GET /identity/v3/domains/99757ed348b847b1b1ba818dcce75f3e => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:43.613510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:43.689719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:43.707415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9a2dc-c805-4f5a-9c3f-dd66261b23c3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:43.740487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:43.741025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:43.741299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:43.741564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:43.742224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:43.743780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c5659-81e7-4cd9-a1c9-dbcaa6ac7740 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:43.840485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 452/886] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:18:43 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:45.977437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56548-630f-4be7-9556-50adbbd0ace0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:45.977990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56548-630f-4be7-9556-50adbbd0ace0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:45.977990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56548-630f-4be7-9556-50adbbd0ace0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:45.979332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 435/887] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:45.984670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:45.985054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:45.985222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:46.003243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:46.004447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:46.069171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 453/888] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:46.337564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a56548-630f-4be7-9556-50adbbd0ace0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:46.435045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a56548-630f-4be7-9556-50adbbd0ace0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:46.468198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:46.468814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:46.469094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:46.469365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] PATH_INFO: `/v3/domains/99757ed348b847b1b1ba818dcce75f3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:46.470460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=99757ed348b847b1b1ba818dcce75f3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:46.471906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20eaf45-89e2-4413-9da3-0b7253d9ed85 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:46.473417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 436/889] 10.222.0.39 () {56 vars in 1183 bytes} [Wed Oct 19 13:18:46 2022] GET /identity/v3/domains/99757ed348b847b1b1ba818dcce75f3e => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:46.479010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:46.537371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:46.573787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1425c-1551-4b4b-85f1-d0d6cd35ee77 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:46.607030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:46.607569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:46.607850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:46.608125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:46.608781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:46.610196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883ef0-fa87-48ab-93bd-a2f92f8e9748 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:46.690077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 454/890] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:18:46 2022] POST /identity/v3/users => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:18:48.805135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11186-e375-4d68-92bb-c651ff0aa29f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:48.805659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11186-e375-4d68-92bb-c651ff0aa29f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:48.805659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c11186-e375-4d68-92bb-c651ff0aa29f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:48.807049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 437/891] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:48.812064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:48.812503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:48.812594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:48.830282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:48.831561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:48.895079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 455/892] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:49.161066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11186-e375-4d68-92bb-c651ff0aa29f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:49.165469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:49.262348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c11186-e375-4d68-92bb-c651ff0aa29f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:49.295455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d4205-6270-434b-b077-a34416205d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:49.296072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d4205-6270-434b-b077-a34416205d9d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:49.296355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d4205-6270-434b-b077-a34416205d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:49.296630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d4205-6270-434b-b077-a34416205d9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:49.306357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d4205-6270-434b-b077-a34416205d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-093d4205-6270-434b-b077-a34416205d9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:49.311235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:49.312479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:18:49.313722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 438/893] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:49.318201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:49.364758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:49.412762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6750aa-175c-4cd3-95e1-c4335d630384 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:49.446060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:49.446672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:49.446952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:49.447222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:49.447872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:49.448168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:49.449601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:49.457226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 456/894] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:49.463674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d4205-6270-434b-b077-a34416205d9d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:49.524460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:49.560479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d4205-6270-434b-b077-a34416205d9d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:49.593725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:49.594335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:49.594650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:49.594927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] PATH_INFO: `/v3/users/f1ecb2c22e664f8c9c16c7719a2ff80d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:49.607536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] RBAC: Authorizing `identity:get_user(user_id=f1ecb2c22e664f8c9c16c7719a2ff80d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:49.608963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:49.610526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 439/895] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:49 2022] GET /identity/v3/users/f1ecb2c22e664f8c9c16c7719a2ff80d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:49.616019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:49.712432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfc2b5-7197-4063-ac91-b87f5b92dd9a None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:49.722103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:49.745277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:49.745897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:49.746182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:49.746498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] PATH_INFO: `/v3/projects/2cf3c61daedf4801bdc01e71dce8c273` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:49.747598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] RBAC: Authorizing `identity:get_project(project_id=2cf3c61daedf4801bdc01e71dce8c273)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:49.749018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222fecc-3490-4463-9aa4-420970144775 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:49.750648 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 457/896] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:49 2022] GET /identity/v3/projects/2cf3c61daedf4801bdc01e71dce8c273 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:49.756353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:49.853069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7b341-ddb6-461c-b25d-8300ea0001ac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:49.875262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:49.886685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:49.887332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:49.887617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:49.887900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] PATH_INFO: `/v3/projects/2cf3c61daedf4801bdc01e71dce8c273/users/f1ecb2c22e664f8c9c16c7719a2ff80d/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:49.889770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] RBAC: Authorizing `identity:create_grant(project_id=2cf3c61daedf4801bdc01e71dce8c273, user_id=f1ecb2c22e664f8c9c16c7719a2ff80d, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:49.891209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53497d-ec44-4e91-ac1b-ef9f0dfbf753 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:50.004956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 440/897] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:18:49 2022] PUT /identity/v3/projects/2cf3c61daedf4801bdc01e71dce8c273/users/f1ecb2c22e664f8c9c16c7719a2ff80d/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:18:52.086995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d91f-93e9-4e91-99ef-e60a349cc49e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:52.087496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d91f-93e9-4e91-99ef-e60a349cc49e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:52.087582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d91f-93e9-4e91-99ef-e60a349cc49e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:52.088906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 458/898] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:18:52.094000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:52.094439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:52.094638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:52.112367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:18:52.113533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:18:52.177554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 441/899] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:18:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:18:52.446977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d91f-93e9-4e91-99ef-e60a349cc49e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:52.545059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d91f-93e9-4e91-99ef-e60a349cc49e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:52.577343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:52.577954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:52.578234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:52.578533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:52.588970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:18:52.593815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:18:52.594988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:18:52.595844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 459/900] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:18:52.601355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:52.696364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ca539-82cb-42fc-825c-0698769d6b91 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:52.710007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:52.731153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:52.731836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:52.732118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:52.732390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:52.733043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:52.733343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:52.734762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:52.742477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 442/901] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:52.748350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:52.787688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:52.846016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de91040-95ae-424b-a892-47e9dbbdcaf1 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:52.878555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:52.879175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:52.879454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:52.879730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] PATH_INFO: `/v3/users/f1ecb2c22e664f8c9c16c7719a2ff80d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:52.880835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ecb2c22e664f8c9c16c7719a2ff80d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:52.882283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:52.883913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 460/902] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Oct 19 13:18:52 2022] GET /identity/v3/users/f1ecb2c22e664f8c9c16c7719a2ff80d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:52.889598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:52.985884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa3b23-313b-4e30-a7a0-a5d84ef9de68 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:53.017251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:53.017865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:53.018145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:53.018412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:53.059148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:53.083134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:53.084562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:53.177432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 443/903] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:18:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:18:53.183454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:53.280992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8814-12d3-4bcd-978d-f17ddf8b8bc8 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:53.287540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:53.312921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:53.313541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:53.313825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:53.314101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] PATH_INFO: `/v3/projects/2cf3c61daedf4801bdc01e71dce8c273` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:53.315258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] RBAC: Authorizing `identity:get_project(project_id=2cf3c61daedf4801bdc01e71dce8c273)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:53.316676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9619040-3a12-49eb-8c22-efa2863082fe None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:53.318263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 461/904] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Oct 19 13:18:53 2022] GET /identity/v3/projects/2cf3c61daedf4801bdc01e71dce8c273 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:18:53.324843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:18:53.395778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:18:53.419768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c8494-10f9-4ca4-aaf0-02762c372648 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:18:53.452196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:18:53.452781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:18:53.453059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:18:53.453330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:18:53.454035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=2cf3c61daedf4801bdc01e71dce8c273, user.id=f1ecb2c22e664f8c9c16c7719a2ff80d) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:18:53.454337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:18:53.455772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e561e-5101-4fb1-93a9-497e40d140c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:18:53.462150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 444/905] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:18:53 2022] GET /identity/v3/role_assignments?user.id=f1ecb2c22e664f8c9c16c7719a2ff80d&scope.project.id=2cf3c61daedf4801bdc01e71dce8c273&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:19:03.321373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=13 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:19:03.459104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=31 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:05.302583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808816a-5b62-436a-937c-bc62f669309d None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:05.305987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808816a-5b62-436a-937c-bc62f669309d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:05.305987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808816a-5b62-436a-937c-bc62f669309d None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:05.305987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 462/906] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:05.309741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:05.310143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:05.310297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:05.328241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:05.329414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:05.393444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 445/907] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:05.660485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808816a-5b62-436a-937c-bc62f669309d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:05.760582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808816a-5b62-436a-937c-bc62f669309d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:05.792942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:05.793562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:05.793840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:05.794110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:05.805559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:05.807005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:05.821377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:05.822396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:05.823609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:05.823609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:05.823609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:05.823609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 463/908] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:05.828871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:05.926236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925a6b8-3381-4892-9b0c-9f6df4e625f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:05.954755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:05.960761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:05.965345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80623987-a824-4c77-9db8-4056cbd3f5ec None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:05.975771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 446/909] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:05.981572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:06.035626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:06.083238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c33247-d395-409e-8b8d-f64c8bf3c469 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:06.115205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:06.115807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:06.116089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:06.116367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:06.117017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:06.118481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363789fd-5906-436a-a041-3dc387334d89 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:06.301165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 464/910] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Oct 19 13:20:05 2022] POST /identity/v3/users => generated 251 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:20:08.425067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5f549-c64b-4891-a0a8-fb9e688dec5f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:08.425605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5f549-c64b-4891-a0a8-fb9e688dec5f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:08.425719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5f549-c64b-4891-a0a8-fb9e688dec5f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:08.427028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 447/911] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:08.432235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:08.432660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:08.432763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:08.451189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:08.452446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:08.520057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 465/912] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:08.788738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5f549-c64b-4891-a0a8-fb9e688dec5f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:08.889156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5f549-c64b-4891-a0a8-fb9e688dec5f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:08.921838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:08.922451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:08.922757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:08.923031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:08.932914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:20:08.938281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:08.946427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:08.947462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 448/913] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:08.948267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:08.984924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:09.042454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac121c5-b69b-46e8-a753-7d8de810597d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.076735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.077901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.078213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.078553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.083404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] RBAC: Adding query filter params (name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:09.083404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.083404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.089035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 466/914] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:09.094903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.144810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:09.193800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5820ed8-c54d-4a95-8b13-30212836fa02 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.226523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.227143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.227422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.227696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.238217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.239715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:09.254026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:09.255060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:09.256153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:09.256153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:09.256153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:09.256153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 449/915] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:09.260986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.355467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:09.358324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2acb11-34e1-49f2-9b56-185fcab8a472 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.392805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.393417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.393704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.393984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.394659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:09.394967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.396354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.408219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 467/916] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:09.414035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.513735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dee51-7914-4e26-aae7-ac51bf1be0a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.515286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:09.546663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.547273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.547548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.547814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.554429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.557214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:20:09.567890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:09.569100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:09.570332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 450/917] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:09.574995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.672966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b6932-47b4-46cb-9272-1b44c5fc986e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.707719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.708326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.708609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.708883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.709604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:09.709910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.711383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.729376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 468/918] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:09.735964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.833503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6a95b-51a1-49da-929a-e60585e4ab0d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:09.867318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:09.867927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:09.868207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:09.868478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:09.879059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:09.880491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:09.894655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:09.895792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:09.897072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:09.897072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:09.897072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:09.897072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 451/919] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:09.901508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:09.920667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:10.000850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db91e7-8a84-4530-bc82-93d010b89644 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:10.035305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:10.035916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:10.036227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:10.036501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:10.037142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:10.037449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:10.038857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:10.050487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 469/920] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:10.056318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:10.072287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:10.154669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dad63-815a-41bb-8741-55d678cc947c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:10.188135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:10.188763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:10.189041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:10.189313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:10.195693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:10.197123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:20:10.208268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:10.213847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:10.214981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 452/921] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:10.215619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:10.290902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:10.312339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93758a68-d5cc-4172-b0f1-e42e0d1a1658 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:10.344141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:10.344738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:10.345019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:10.345294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:10.345976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:10.346321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:10.347771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36205f-4db4-493a-8fbe-95ccf9d207eb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:10.360315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 470/922] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:20:10.366606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:10.441525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:10.464776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54c0c-31b7-46e4-8906-e6ac1042abc7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:10.498042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:10.498729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:10.499018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:10.499291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/99786bcc97fa4b7eb9141fb3d509c28c/roles/1cc7eebbaac44039a8da36599392cd1e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:10.512773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=99786bcc97fa4b7eb9141fb3d509c28c, role_id=1cc7eebbaac44039a8da36599392cd1e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:10.514190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a44f0e-92e4-4323-ace2-04d554af8e16 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:10.587724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 453/923] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:20:10 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/99786bcc97fa4b7eb9141fb3d509c28c/roles/1cc7eebbaac44039a8da36599392cd1e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:20:12.706738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e0e80-dd5b-472f-a31c-c552bdef53f6 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:12.707319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e0e80-dd5b-472f-a31c-c552bdef53f6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:12.707319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e0e80-dd5b-472f-a31c-c552bdef53f6 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:12.708720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 471/924] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:12.714041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:12.714524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:12.714662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:12.732226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:12.733400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:12.798329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 454/925] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:13.069890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e0e80-dd5b-472f-a31c-c552bdef53f6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.169773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e0e80-dd5b-472f-a31c-c552bdef53f6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.205704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.206336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.206647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.206925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.213606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:13.217469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:20:13.222359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:13.223957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:13.224889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 472/926] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:13.229856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.283424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:13.327604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6914-7d4a-4c3c-8f3b-c228683bff86 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.360402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.360994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.361276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.361544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.362197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:13.362495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:13.363876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:13.371322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 455/927] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:13.378041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.476851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c90b03-8a26-4700-8ef2-04673db10644 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.510524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.511160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.511444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.511720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.522194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:13.523743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:13.538283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:13.539351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:13.540272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:13.540272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:13.540272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:13.540272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 473/928] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 13:20:13.545249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.642921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e0a8e-3641-41c3-bf68-b4e63aff86ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.658010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:13.674835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.676114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.676114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.676114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.676561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:13.676860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:13.678208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:13.689798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 456/929] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:13.695538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.746027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:13.795114 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781035a-7bb9-4d87-920a-2715bf6b7c9c None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.827934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.828551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.828831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.829104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.836502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:13.837935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:20:13.849640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:13.850842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:13.852267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 474/930] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:13.856585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:13.954760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29657b8e-2de6-4c17-a4ad-628ce8c283dd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:13.988348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:13.988938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:13.989219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:13.989491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:13.990206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:13.990539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:13.991941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.008958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 457/931] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Oct 19 13:20:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:14.015362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:14.113833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c412cc-812e-402b-ab24-c401a595e039 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:14.123818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:14.146595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:14.147200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:14.147487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:14.147761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:14.214003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:14.215468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.310084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 475/932] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:20:14.315908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:14.349858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:14.413773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05461-30d0-4598-85e4-33637b87da55 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:14.446611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:14.447240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:14.447521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:14.447818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:14.457976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:14.460119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:14.475292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:14.476461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:14.477707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:14.477707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:14.477707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:14.477707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 458/933] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:14.482108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:14.494277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:14.579852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8fa83-b092-4262-b969-c149db4f1877 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:14.614411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:14.615062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:14.615341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:14.615612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:14.616245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:14.616638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:14.618052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.629493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 476/934] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:14.635174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:14.719101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:14.733643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61a82-6989-42ee-a33a-173a6b9e17cf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:14.766252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:14.766912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:14.767193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:14.767464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:14.773965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:14.775413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:20:14.786302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:14.787390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:14.788677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 459/935] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:14.793272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:14.861370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:14.891501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce542a-f97f-49e4-922e-1deba5818dc5 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:14.923639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:14.924240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:14.924523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:14.924795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:14.925483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:14.925790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:14.927233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bc542-c876-4f51-b337-4a484947f738 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:14.940783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 477/936] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:14.946934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:15.046555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2f207-1e4e-4e95-8dcb-579f3920fc67 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:15.078327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:15.078951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:15.079232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:15.079507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:15.080251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] RBAC: Adding query filter params (role.id=1cc7eebbaac44039a8da36599392cd1e, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=99786bcc97fa4b7eb9141fb3d509c28c) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:15.080813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:15.082232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd364f18-a390-4e00-8514-862b158bc85a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:15.087942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 460/937] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:20:14 2022] GET /identity/v3/role_assignments?user.id=99786bcc97fa4b7eb9141fb3d509c28c&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=1cc7eebbaac44039a8da36599392cd1e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:17.234089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012554-67a4-437a-be57-391274f38e7f None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:17.234692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012554-67a4-437a-be57-391274f38e7f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:17.234692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc012554-67a4-437a-be57-391274f38e7f None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:17.235998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 478/938] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:17.241182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367516b0-263f-4369-9321-9536ce203325 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:17.241578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367516b0-263f-4369-9321-9536ce203325 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:17.241729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367516b0-263f-4369-9321-9536ce203325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:17.265801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-367516b0-263f-4369-9321-9536ce203325 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:17.265801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-367516b0-263f-4369-9321-9536ce203325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:17.265801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:17.325815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 461/939] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:20:17.594365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc012554-67a4-437a-be57-391274f38e7f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:17.693010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc012554-67a4-437a-be57-391274f38e7f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:17.721996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:17.727555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:17.728172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:17.728450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:17.728720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:17.738807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:20:17.743962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:17.745129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:17.746082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 479/940] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:17.751339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367516b0-263f-4369-9321-9536ce203325 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:17.850321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367516b0-263f-4369-9321-9536ce203325 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:17.882472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:17.883133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:17.883410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:17.883677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:17.884326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:17.884624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:17.886115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:17.894442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 462/941] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:17.900216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:17.999887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984871e9-74af-4d6f-8e2e-ebf924b8c455 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.034060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.034714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.035040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.035349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.044935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.046363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:18.060551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:18.062214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:18.063356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:18.063356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:18.063356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:18.063356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 480/942] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:18.068114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.071051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:18.166885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e65ed-eabb-448f-b346-eceef86e56a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.199939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.200542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.200826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.201102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.202358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:18.202697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.204120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.215890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 463/943] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:18.221743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.264149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:18.319512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a09e95-7f18-4dcc-b371-7f744abad5bc None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.352245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.352852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.353130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.353400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.361369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.363484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:20:18.375213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:18.376773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:18.377824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 481/944] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:18.382343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.442370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:18.478233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e451b-f088-4be5-82a2-06ddf60bd986 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.510832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.511418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.511694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.511963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.512718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:18.513015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.514441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.530658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 464/945] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:18.536548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.635505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6e263-eec4-45a7-9c16-bbd301b3f385 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.637012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:18.668024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.668647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.668928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.669203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.680357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.681787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:18.696697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:18.697837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:18.698807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:18.698807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:18.698807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:18.698807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 482/946] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:18.703524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.801627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cabb6-a8c8-4942-8a07-f1bb64ba99ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.821683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:18.835229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.835861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.836143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.836415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.837047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:18.837350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.838760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:18.850044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 465/947] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:18.855794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:18.954645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640fd4-64f0-44f6-93be-f4af4bd7e2b4 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:18.987372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:18.988014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:18.988291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:18.988560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:18.994996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:18.996457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:20:19.008111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:19.009201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:19.010388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 483/948] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:19.015712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:19.114364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc7ba7-627a-4922-80b6-267f53a24d69 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:19.147043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:19.147636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:19.147910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:19.148179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:19.148858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:19.149168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:19.150597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416f1c-91e5-4c4a-96f3-03b07f204dd0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:19.162127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 466/949] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:19.168217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:19.184560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:19.266225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d6d8e-42db-4894-a3de-03eb6b44b8f6 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:19.299035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:19.299677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:19.299957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:19.300376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] PATH_INFO: `/v3/projects/60cab2a8445f4898bae2696caebbb468/users/99786bcc97fa4b7eb9141fb3d509c28c/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:19.302223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] RBAC: Authorizing `identity:create_grant(project_id=60cab2a8445f4898bae2696caebbb468, user_id=99786bcc97fa4b7eb9141fb3d509c28c, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:19.303690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a30d8-ea06-4ef9-83d6-798308b44436 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:19.422123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 484/950] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Oct 19 13:20:19 2022] PUT /identity/v3/projects/60cab2a8445f4898bae2696caebbb468/users/99786bcc97fa4b7eb9141fb3d509c28c/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:20:21.550210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3225d-64ad-462c-bbb1-105fa7f36225 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:21.550807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3225d-64ad-462c-bbb1-105fa7f36225 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:21.550807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3225d-64ad-462c-bbb1-105fa7f36225 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:21.552145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 467/951] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:21.557407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:21.557849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:21.557969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:21.575991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:21.577654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:21.643500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 485/952] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:21.914133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3225d-64ad-462c-bbb1-105fa7f36225 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:21.956240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.014713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3225d-64ad-462c-bbb1-105fa7f36225 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.048464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.049080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.049438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.049715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.059391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 13:20:22.064247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:22.065404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:22.066645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 468/953] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Oct 19 13:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:22.071443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.074530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.168175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d06ab-c3bc-4b69-810a-e2b10145d33b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.200084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:22.204912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:22.212056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 486/954] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:22.217994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.313627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.316370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146a80f-1dd5-4ea4-a396-25d968406415 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.351376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.352038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.352318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.352591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.362967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:22.364434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:22.379616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:22.381180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:22.382136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:22.382136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:22.382136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:22.382136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 469/955] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:22.386638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.450409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.484387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb028a-d49d-4e7f-95a0-2668998bbca9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.515818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.516412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.516688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.516957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.517634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:22.517934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:22.519331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:22.531564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 487/956] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:22.537399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.635449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f27d7d-84b1-47b3-9b2a-8ad40440c8f7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.668400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.669018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.669298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.669590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.677347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:22.678799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 13:20:22.689644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:22.690808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:22.692012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 470/957] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:22.696551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.793703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb916fc5-f889-4670-bbf1-86b8b9a50ea9 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.818937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.828029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.828631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.828913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.829190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:22.829924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:22.830520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:22.831968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:22.849085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 488/958] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:22.854832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:22.869921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:22.952812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf7ac0-40e1-42da-9c2a-d4dbddf5362e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:22.986324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:22.986952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:22.987235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:22.987509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.054094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.055580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.068542 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:23.151282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 471/959] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Oct 19 13:20:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 13:20:23.157179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:23.255465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27bd-edd0-4963-9ade-83eaeedaba8b None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:23.288544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:23.289192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:23.289498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:23.289773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.300134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.301697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 13:20:23.315830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:23.316953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 13:20:23.317879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 13:20:23.317879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 13:20:23.317879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:23.317879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 489/960] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:23.322576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:23.420116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84091590-6a90-4bf1-ac09-36b4a0faa5ce None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:23.432956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:23.452974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:23.453593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:23.453873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:23.454145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.454813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:23.455367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.456710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.468574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 472/961] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Oct 19 13:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:23.474288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:23.509281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:23.571740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea322e7-d6e5-42bf-aea6-4d5af04f3bc2 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:23.605491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:23.606117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:23.606398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:23.606706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.613243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.614778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 13:20:23.625616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:23.626705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: ERROR keystone.server.flask.application Oct 19 13:20:23.627990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 490/962] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Oct 19 13:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:23.632709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:23.729788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768e25-254f-482e-a8ba-a645286808cd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:23.761689 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:23.762274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:23.762575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:23.762850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.763522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:23.763821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.765748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4334b0a-301a-45a5-b3b6-ba16da6922be None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.776805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 473/963] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Oct 19 13:20:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:23.783407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:23.876728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:23.882364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca95c4f-fac3-4c66-b715-f11f4332d812 None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:23.916265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:23.916859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:23.917136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:23.917407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:23.918156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] RBAC: Adding query filter params (role.id=821e0e3786404ca6a02982ae9bd496a2, scope.project.id=60cab2a8445f4898bae2696caebbb468, user.id=99786bcc97fa4b7eb9141fb3d509c28c) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:23.918464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:23.919915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690fb8a-1867-4e29-a652-0108d11fae7e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:23.925566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 491/964] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Oct 19 13:20:23 2022] GET /identity/v3/role_assignments?user.id=99786bcc97fa4b7eb9141fb3d509c28c&scope.project.id=60cab2a8445f4898bae2696caebbb468&role.id=821e0e3786404ca6a02982ae9bd496a2 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:26.085200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4325-8fff-44ce-a99b-794f15b9d2d8 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:26.085711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4325-8fff-44ce-a99b-794f15b9d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:26.085825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4325-8fff-44ce-a99b-794f15b9d2d8 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:26.087214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 474/965] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:26.092626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:26.093080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:26.093175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:26.111588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:26.112858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:26.177617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 492/966] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:26.452361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4325-8fff-44ce-a99b-794f15b9d2d8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:26.483958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:26.553396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4325-8fff-44ce-a99b-794f15b9d2d8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:26.587487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:26.588107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:26.588385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:26.588655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:26.589230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:26.590616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:26.599529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:26.604694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:26.605752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 475/967] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:20:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:26.606907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:26.695264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:26.705460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2b86-5527-41ba-a466-c2a184442049 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:26.738338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:26.738984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:26.739263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:26.739530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:26.740161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:26.740483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:26.741839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad5a-00d9-4672-8156-e1b4d7b0893f None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:26.746855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 493/968] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Oct 19 13:20:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:26.753169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:26.844962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:26.850718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af57a-1bac-4255-ae49-99c192bfc72f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:26.885172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:26.885771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:26.886046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:26.886316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:26.886968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:26.887266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:26.888620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79f788-d280-415e-9a19-b738355e45f7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:26.892993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 476/969] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Oct 19 13:20:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:29.038283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe07a70-50fa-4b41-a1bb-48ebfc4d20ad None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:29.038831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe07a70-50fa-4b41-a1bb-48ebfc4d20ad None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:29.038926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe07a70-50fa-4b41-a1bb-48ebfc4d20ad None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:29.040200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 494/970] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:29.045913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02831-88e1-43b3-a6e3-efdc235876cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:29.046327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02831-88e1-43b3-a6e3-efdc235876cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:29.046445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c02831-88e1-43b3-a6e3-efdc235876cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:29.064826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d2c02831-88e1-43b3-a6e3-efdc235876cd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:29.066002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2c02831-88e1-43b3-a6e3-efdc235876cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:29.131198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 477/971] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 13:20:29.403734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe07a70-50fa-4b41-a1bb-48ebfc4d20ad None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:29.503804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe07a70-50fa-4b41-a1bb-48ebfc4d20ad None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:29.537236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:29.537835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:29.538119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:29.538397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:29.539089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:29.540545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c435b66-7d92-43e0-961f-628b993a9881 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:29.599131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 495/972] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Oct 19 13:20:29 2022] POST /identity/v3/services => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:20:31.730743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b2f6-2298-4738-bc22-b62ba0a435df None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:31.731327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b2f6-2298-4738-bc22-b62ba0a435df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:31.731327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b2f6-2298-4738-bc22-b62ba0a435df None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:31.733174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 478/973] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:31.738501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:31.738892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:31.739082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:31.757088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:31.758259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:31.767303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:31.830605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 496/974] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:32.098375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b2f6-2298-4738-bc22-b62ba0a435df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:32.186967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:32.196621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b2f6-2298-4738-bc22-b62ba0a435df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:32.228841 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:32.229455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:32.229733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:32.230002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:32.230596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:32.231938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:32.240935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:32.247546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:32.248580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 479/975] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:20:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:32.249392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:32.344692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c209a16-cedb-47a0-85b5-8fa4f85e7360 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:32.377526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:32.378119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:32.378395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:32.378697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:32.379319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:32.379618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:32.381038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:32.385992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 497/976] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Oct 19 13:20:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:20:32.392018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:32.490115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf5c32-1ca0-4819-a0f0-ad40b418fa14 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:32.522023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:32.522644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:32.522930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:32.523237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:32.525410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] RBAC: Adding query filter params (interface=public, service_id=400a933695d04b3ebe4bd0d0a87e8ef4, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:32.525410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:32.525598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322f11a-4aff-4038-ad29-4363e2beaefa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:32.530897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 480/977] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Oct 19 13:20:32 2022] GET /identity/v3/endpoints?service_id=400a933695d04b3ebe4bd0d0a87e8ef4&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:32.536617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:32.540480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:32.633862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852fbdf-8860-4690-bef7-570eb98d709e None admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:32.669300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:32.669905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:32.670188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:32.670460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:32.671099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:32.672513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb882c-44b7-49da-a87c-00a40d4e9049 None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:32.679331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 498/978] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Oct 19 13:20:32 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:20:34.797406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1f602-542a-4971-96e1-8b4bd026196c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:34.797961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1f602-542a-4971-96e1-8b4bd026196c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:34.797961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1f602-542a-4971-96e1-8b4bd026196c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:34.799339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 481/979] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:20:34.804471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:34.804894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:34.805022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:34.823061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:20:34.824236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:20:34.884317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:34.890042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 499/980] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:20:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 13:20:35.154035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1f602-542a-4971-96e1-8b4bd026196c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:35.173461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:35.255236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1f602-542a-4971-96e1-8b4bd026196c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:35.288730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:35.289387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:35.289667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:35.289935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:35.290500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:35.291876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 13:20:35.301060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 13:20:35.302428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Oct 19 13:20:35.303613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 482/981] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Oct 19 13:20:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 13:20:35.308232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:35.405765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcddc92-af4a-4b9c-af19-2c2e64b6f141 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:35.439523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:35.440156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:35.440438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:35.440713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] PATH_INFO: `/v3/services` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:35.441340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:20:35.441700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:35.443126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28934648-0708-44ce-9306-245a8eb0c2fb None admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:35.448355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 500/982] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Oct 19 13:20:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:20:35.454086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:20:35.534049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:35.552351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716123-340d-4d7d-b2a9-02c4e78a001b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:20:35.585498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:20:35.586131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:20:35.586414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:20:35.586720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:20:35.587378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:20:35.588747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813664e-62a1-4086-a48d-39bd997a462c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:20:35.672805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 483/983] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:20:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:20:45.445181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=69 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:20:45.668312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:27.527098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb721d-ad01-4075-97d3-f9e01a1ad6ae None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:27.527814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb721d-ad01-4075-97d3-f9e01a1ad6ae None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:27.528098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb721d-ad01-4075-97d3-f9e01a1ad6ae None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:27.533783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 501/984] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:27.540102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414daf-80e5-4e68-8ee8-aaab82d95f1f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:27.545359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414daf-80e5-4e68-8ee8-aaab82d95f1f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:27.545359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40414daf-80e5-4e68-8ee8-aaab82d95f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:27.694845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-40414daf-80e5-4e68-8ee8-aaab82d95f1f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:27.697953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-40414daf-80e5-4e68-8ee8-aaab82d95f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:27.877559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 484/985] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:28.174357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:28.174831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:28.175263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:28.176932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 502/986] 10.222.0.39 () {54 vars in 964 bytes} [Wed Oct 19 13:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:28.183305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4d4a-73d2-49ed-b487-210502d16117 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:28.183792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4d4a-73d2-49ed-b487-210502d16117 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:28.184328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4d4a-73d2-49ed-b487-210502d16117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:28.276639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d8dd4d4a-73d2-49ed-b487-210502d16117 None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:28.278196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d8dd4d4a-73d2-49ed-b487-210502d16117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:28.301943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:28.374707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:28.376710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:28.382142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 485/987] 10.222.0.39 () {58 vars in 1061 bytes} [Wed Oct 19 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:26:28.499409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d010d-f45f-4395-a7eb-ffc41c15e628 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:28.589218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:28.590681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:28.591113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:28.591596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:28.593555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:28.657146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:28.732205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:28.733995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a068d-3e74-4930-8a21-13afd4928bc0 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:28.773862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 503/988] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:31.267244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:31.267696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:31.267797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:31.269414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 486/989] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:31.274418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365a1c-c5a0-4af9-b983-588cf4696e3c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:31.274868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365a1c-c5a0-4af9-b983-588cf4696e3c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:31.274991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365a1c-c5a0-4af9-b983-588cf4696e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:31.344705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-16365a1c-c5a0-4af9-b983-588cf4696e3c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:31.345893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-16365a1c-c5a0-4af9-b983-588cf4696e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:31.444023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 504/990] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:31.726594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:31.728145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:31.829941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee18756-189e-4425-98e6-9ba64d1173d5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:31.905977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:31.908327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:31.908940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:31.909219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:31.909493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:31.911118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:32.019813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:32.021264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8dc7a4-aef9-45cd-b4a1-be90f2fc2151 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:32.039456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 487/991] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:34.618609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:34.619144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:34.619144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:34.620515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 505/992] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:34.625814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3234c-9073-4348-b57d-3740db80e3a3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:34.626231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3234c-9073-4348-b57d-3740db80e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:34.626364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3234c-9073-4348-b57d-3740db80e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:34.692422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3da3234c-9073-4348-b57d-3740db80e3a3 None None] MFA Rules not processed for user `aeec84e937e04e0cbd930c7b84cb139d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:34.693629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3da3234c-9073-4348-b57d-3740db80e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:34.744892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:34.788116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 488/993] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:26:35.076252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:35.077783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:35.087528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:35.179300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd3472-f290-4305-865a-4891a99bfde4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:35.252821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:35.253466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:35.253796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:35.254017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:35.255675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:35.316479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:35.363105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:35.364527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d20fa-4ab6-4e0a-975a-deab00726f93 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:35.383579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 506/994] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:38.393272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de5790-c745-4097-a773-563e7daac223 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:38.393835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de5790-c745-4097-a773-563e7daac223 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:38.393932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de5790-c745-4097-a773-563e7daac223 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:38.395231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 489/995] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:38.400457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b3b1-0241-4157-95e1-6c11cf5ea8b4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:38.400834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b3b1-0241-4157-95e1-6c11cf5ea8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:38.401009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b3b1-0241-4157-95e1-6c11cf5ea8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:38.465569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-51f2b3b1-0241-4157-95e1-6c11cf5ea8b4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:38.466802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-51f2b3b1-0241-4157-95e1-6c11cf5ea8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:38.561433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 507/996] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:38.848221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de5790-c745-4097-a773-563e7daac223 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:38.849757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f2de5790-c745-4097-a773-563e7daac223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:38.952023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de5790-c745-4097-a773-563e7daac223 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:39.026445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:39.027066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:39.027342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:39.027607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:39.029171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:39.035936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:39.137993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:39.139387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3102c-6cd5-41bf-91e8-6c9d96709145 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:39.156897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 490/997] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:42.342572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:42.343115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:42.343115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:42.344714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 508/998] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:42.350601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b082-4ca8-40ad-b2d5-05e960928d4d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:42.351031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b082-4ca8-40ad-b2d5-05e960928d4d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:42.351292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43b082-4ca8-40ad-b2d5-05e960928d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:42.417693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ff43b082-4ca8-40ad-b2d5-05e960928d4d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:42.418957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff43b082-4ca8-40ad-b2d5-05e960928d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:42.483038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:42.526748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 491/999] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:42.806738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:42.808307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:42.829046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:42.912334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c55f8-e83f-4e41-8089-de38521bfa74 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:42.985487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:42.986099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:42.986381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:42.986691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:42.988391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:43.060760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:43.098887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:43.100358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4c63c-2cd7-4a9e-884c-ad1ee4122eb5 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:43.118060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 509/1000] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:45.313441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:45.313982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:45.314062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:45.315482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 492/1001] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:45.321398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ca43f-fcbf-4e25-8c5d-4ae706f59286 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:45.321890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ca43f-fcbf-4e25-8c5d-4ae706f59286 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:45.321979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ca43f-fcbf-4e25-8c5d-4ae706f59286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:45.389796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1f9ca43f-fcbf-4e25-8c5d-4ae706f59286 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:45.391041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1f9ca43f-fcbf-4e25-8c5d-4ae706f59286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:45.499109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 510/1002] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:45.774991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:45.776523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:45.880482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed2d2c-bef7-4d6c-b0e7-effe79ef4756 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:45.952477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:45.953074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:45.953349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:45.953619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:45.955258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:45.972497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:46.066483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:46.067987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad977d-532e-4c81-bb3b-ac20838742d1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:46.086080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 493/1003] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:48.294416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833133c6-6c35-4848-86d2-bdd48cd5bbd7 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:48.294987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833133c6-6c35-4848-86d2-bdd48cd5bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:48.295083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833133c6-6c35-4848-86d2-bdd48cd5bbd7 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:48.296406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 511/1004] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:48.302116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ffc8-a35c-4d99-8122-9c189329a82b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:48.302527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ffc8-a35c-4d99-8122-9c189329a82b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:48.302720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ffc8-a35c-4d99-8122-9c189329a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:48.369175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b420ffc8-a35c-4d99-8122-9c189329a82b None None] MFA Rules not processed for user `aeec84e937e04e0cbd930c7b84cb139d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:48.370318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b420ffc8-a35c-4d99-8122-9c189329a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:48.433132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:48.465760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 494/1005] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:48.769492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:48.769784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:48.770066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:48.771572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 512/1006] 10.222.0.39 () {54 vars in 964 bytes} [Wed Oct 19 13:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:48.777523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ea6fb-8e41-4203-8e27-38bb229b28cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:48.777884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ea6fb-8e41-4203-8e27-38bb229b28cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:48.778239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ea6fb-8e41-4203-8e27-38bb229b28cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:48.859410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f72ea6fb-8e41-4203-8e27-38bb229b28cd None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:48.860627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f72ea6fb-8e41-4203-8e27-38bb229b28cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:48.940224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 495/1007] 10.222.0.39 () {58 vars in 1061 bytes} [Wed Oct 19 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:26:48.949815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:48.951428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:48.983891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:49.059400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324d3d9-2bda-48f7-b541-ef9e89b74f37 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:49.132121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:49.132724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:49.132999 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:49.133270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:49.134880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:49.211501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:49.243420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:49.244865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f299f91-0da5-4693-8fea-2647d821dfd0 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:49.262516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 513/1008] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:52.133500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:52.134019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:52.134126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:52.135439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 496/1009] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:52.140496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8135c3-aff4-4297-9240-f8a35d4fa118 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:52.140909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8135c3-aff4-4297-9240-f8a35d4fa118 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:52.141048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8135c3-aff4-4297-9240-f8a35d4fa118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:52.207428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ef8135c3-aff4-4297-9240-f8a35d4fa118 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:52.208611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ef8135c3-aff4-4297-9240-f8a35d4fa118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:52.302186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 514/1010] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:52.585853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:52.587425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:52.656917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:52.691206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85100-a200-40d4-922f-69f3bf84901b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:52.763421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:52.764018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:52.764297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:52.764566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:52.766148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:52.874065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:52.875443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa716b-d3e6-4f0c-a092-1d6571fb68df service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:52.885656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:52.893678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 497/1011] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:55.063300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:55.063826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:55.063918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:55.065197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 515/1012] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:55.070457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a516-2577-4ca9-ba20-8f974974ffce None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:55.070928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a516-2577-4ca9-ba20-8f974974ffce None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:55.071022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a516-2577-4ca9-ba20-8f974974ffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:55.135937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f607a516-2577-4ca9-ba20-8f974974ffce None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:55.137132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f607a516-2577-4ca9-ba20-8f974974ffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:55.233505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 498/1013] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:55.521126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:55.522711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:55.563124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:55.625845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b97f4-2336-466e-af6d-16c5b28c9f52 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:55.696686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:55.697283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:55.697559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:55.697827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:55.699699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:55.780644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:55.805782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:55.807300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956a405-6373-4b46-8b31-d1c361745cb1 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:55.825460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 516/1014] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:26:58.640470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:58.641088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:58.641088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:58.642356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 499/1015] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:26:58.647579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdf0a3-d1a6-4cb1-a204-0313cd7a3388 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:58.648026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdf0a3-d1a6-4cb1-a204-0313cd7a3388 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:58.648135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdf0a3-d1a6-4cb1-a204-0313cd7a3388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:58.713456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1ecdf0a3-d1a6-4cb1-a204-0313cd7a3388 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:26:58.714691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1ecdf0a3-d1a6-4cb1-a204-0313cd7a3388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:58.810011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 517/1016] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:26:59.093803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:26:59.095369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:59.171852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:26:59.198822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4055ca-0642-43bd-9165-9a803a27c3fc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:26:59.271701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:26:59.272229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:26:59.272504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:26:59.272773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:26:59.274339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:26:59.380242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:26:59.381679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b870d2-fbc4-4309-bc7c-b6ec5b38f0d6 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:26:59.399216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 500/1017] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:26:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:03.735250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:03.735796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:03.735796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:03.737147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 518/1018] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:03.742458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2600-f001-449e-8034-8aa42e8d021a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:03.742902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2600-f001-449e-8034-8aa42e8d021a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:03.743187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e2600-f001-449e-8034-8aa42e8d021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:03.748752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:03.811788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-778e2600-f001-449e-8034-8aa42e8d021a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:03.813006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-778e2600-f001-449e-8034-8aa42e8d021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:03.910266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 501/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:04.192766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:04.194359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:04.244326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:04.300164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92d02-719c-44d0-b09e-b69d5ab2e491 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:04.376038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:04.376651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:04.376931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:04.377693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:04.379332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:04.474023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:04.490261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:04.491846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed997e6-a191-42cf-8c26-9ea15eb2c3da service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:04.511033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 519/1020] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:07.600369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:07.600891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:07.601034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:07.602455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 502/1021] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:07.608046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132db39-7487-4e1b-9781-a0771e79fdd0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:07.608523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132db39-7487-4e1b-9781-a0771e79fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:07.608627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132db39-7487-4e1b-9781-a0771e79fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:07.680921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d132db39-7487-4e1b-9781-a0771e79fdd0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:07.682127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d132db39-7487-4e1b-9781-a0771e79fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:07.780635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 520/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:08.063570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:08.065104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:08.152242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:08.169523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2bdc6-20da-4cd7-9994-7de203edf92d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:08.244843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:08.245451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:08.245732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:08.246004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:08.247647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:08.376649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:08.378143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dbbc9-34be-4fe0-8e77-6370fe55e89e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:08.397984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 503/1023] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:12.721096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:12.721671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:12.721671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:12.723521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 521/1024] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:12.729143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087ac-41fc-4d0a-8520-4ff4f0ae0f03 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:12.729594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087ac-41fc-4d0a-8520-4ff4f0ae0f03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:12.729704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29087ac-41fc-4d0a-8520-4ff4f0ae0f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:12.746631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:12.801053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d29087ac-41fc-4d0a-8520-4ff4f0ae0f03 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:12.802395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d29087ac-41fc-4d0a-8520-4ff4f0ae0f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:12.903991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 504/1025] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:13.188236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:13.189830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:13.247787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:13.293700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef36829-6542-49b3-a49e-8033c88ad053 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:13.364500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:13.365093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:13.365369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:13.365639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:13.367227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:13.469839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:13.473350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:13.476596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0167ec8-9c4e-403a-a932-1ca82fd02a66 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:13.492101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 522/1026] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:18.388306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f81f1-8cf9-4c3d-b12b-39528000ccd3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:18.389060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f81f1-8cf9-4c3d-b12b-39528000ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:18.389510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f81f1-8cf9-4c3d-b12b-39528000ccd3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:18.391314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 505/1027] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:27:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:18.397094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69680a-c290-4dbe-a030-f62a6f234d83 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:18.397519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69680a-c290-4dbe-a030-f62a6f234d83 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:18.397664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69680a-c290-4dbe-a030-f62a6f234d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:18.517591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cf69680a-c290-4dbe-a030-f62a6f234d83 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:18.526408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cf69680a-c290-4dbe-a030-f62a6f234d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:18.674699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 523/1028] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:22.902418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=59 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:22.929402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d36db-49a4-4013-ae2e-8051f63a498e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:22.929742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d36db-49a4-4013-ae2e-8051f63a498e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:22.929951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d36db-49a4-4013-ae2e-8051f63a498e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:22.931504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 506/1029] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:22.937040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad6b7-39a3-45c3-a5f7-7c07ed92f9e2 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:22.937867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad6b7-39a3-45c3-a5f7-7c07ed92f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:22.938437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad6b7-39a3-45c3-a5f7-7c07ed92f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:23.008501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:23.038299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-930ad6b7-39a3-45c3-a5f7-7c07ed92f9e2 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:23.039885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-930ad6b7-39a3-45c3-a5f7-7c07ed92f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:23.168094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 524/1030] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:24.340248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1a5bc-fbf7-4960-afee-0a5e0d2d08a6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:24.340680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1a5bc-fbf7-4960-afee-0a5e0d2d08a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:24.340780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1a5bc-fbf7-4960-afee-0a5e0d2d08a6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:24.342147 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 507/1031] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:24.348256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708815d-0179-4a2e-a15e-022af92c1da6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:24.348704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708815d-0179-4a2e-a15e-022af92c1da6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:24.348837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708815d-0179-4a2e-a15e-022af92c1da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:24.426920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1708815d-0179-4a2e-a15e-022af92c1da6 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:24.428137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1708815d-0179-4a2e-a15e-022af92c1da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:24.470695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:24.535766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 525/1032] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:28.028027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4461-88c0-4bc0-aa5d-f94bcdcb48a6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:28.028929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4461-88c0-4bc0-aa5d-f94bcdcb48a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:28.029404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4461-88c0-4bc0-aa5d-f94bcdcb48a6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:28.031585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 508/1033] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:28.039526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ecad-a910-437d-a66c-a2b9c60dae4d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:28.040748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ecad-a910-437d-a66c-a2b9c60dae4d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:28.040748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ecad-a910-437d-a66c-a2b9c60dae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:28.137241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4035ecad-a910-437d-a66c-a2b9c60dae4d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:28.139310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4035ecad-a910-437d-a66c-a2b9c60dae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:28.250634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 526/1034] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:28.547337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:28.548212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:28.550015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:28.552881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 509/1035] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:27:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:28.561561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef613-a6ff-4022-9884-372e3921c970 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:28.562179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef613-a6ff-4022-9884-372e3921c970 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:28.562591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef613-a6ff-4022-9884-372e3921c970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:28.580497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:28.644910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5c5ef613-a6ff-4022-9884-372e3921c970 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:28.646113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c5ef613-a6ff-4022-9884-372e3921c970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:28.781882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 527/1036] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:28.782678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:28.785012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:28.946644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac68ee4-a972-441f-b8ef-c9edd23cc2c3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:29.034162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:29.075956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:29.076638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:29.076919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:29.077276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:29.078883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:29.172569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:29.174141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca35ee-4834-4274-b617-54f046197381 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:29.194702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 510/1037] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:27:31.572902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a265-a763-4312-a57d-f557c265174a None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:31.573412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a265-a763-4312-a57d-f557c265174a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:31.573517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03a265-a763-4312-a57d-f557c265174a None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:31.574830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 528/1038] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:31.580098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92684-2fb3-4eb1-81d1-32cd4f92d643 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:31.580540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92684-2fb3-4eb1-81d1-32cd4f92d643 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:31.580719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92684-2fb3-4eb1-81d1-32cd4f92d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:31.650154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-55b92684-2fb3-4eb1-81d1-32cd4f92d643 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:31.651297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55b92684-2fb3-4eb1-81d1-32cd4f92d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:31.715986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:31.777920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 511/1039] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:32.072863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03a265-a763-4312-a57d-f557c265174a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:32.074385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cb03a265-a763-4312-a57d-f557c265174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:32.153285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:32.251873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03a265-a763-4312-a57d-f557c265174a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:32.409065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:32.409888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:32.410453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:32.410915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:32.412654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:32.450640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:32.512151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:32.513687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e080519-56ce-4eb6-aaae-420497469284 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:32.532694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 529/1040] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:35.181914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8728c-7187-4fad-935f-8e7be8407902 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:35.182837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8728c-7187-4fad-935f-8e7be8407902 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:35.183280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8728c-7187-4fad-935f-8e7be8407902 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:35.184784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 512/1041] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:35.190168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dfd2f-40bb-452e-ac35-5ba770be72d9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:35.190919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dfd2f-40bb-452e-ac35-5ba770be72d9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:35.191364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dfd2f-40bb-452e-ac35-5ba770be72d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:35.263634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6c9dfd2f-40bb-452e-ac35-5ba770be72d9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:35.265076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6c9dfd2f-40bb-452e-ac35-5ba770be72d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:35.365796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:35.374754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 530/1042] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:35.657199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:35.657506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:35.657770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:35.659944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 513/1043] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:35.667260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2813865-1de2-4ad5-a0c7-713f9df05ddb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:35.667649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2813865-1de2-4ad5-a0c7-713f9df05ddb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:35.668022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2813865-1de2-4ad5-a0c7-713f9df05ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:35.749360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b2813865-1de2-4ad5-a0c7-713f9df05ddb None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:35.750724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b2813865-1de2-4ad5-a0c7-713f9df05ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:35.873052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 531/1044] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:35.882321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:35.884279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:36.047868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9615df8-9ae7-4ba3-97e4-99b6fca747e4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:36.070188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:36.171172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:36.171895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:36.172237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:36.172571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:36.174229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:36.270014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:36.271735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc27093-731d-4d60-9d4f-99d2ea1b2ebb service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:36.295200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 514/1045] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:27:36.392189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484d91-5fc3-48d0-977e-eafa1ad14f38 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:36.392720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484d91-5fc3-48d0-977e-eafa1ad14f38 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:36.392720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7484d91-5fc3-48d0-977e-eafa1ad14f38 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:36.394163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 532/1046] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:36.399762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eed41-2c2d-48bf-ac2a-99d5ff7bdee4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:36.400222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eed41-2c2d-48bf-ac2a-99d5ff7bdee4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:36.400587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eed41-2c2d-48bf-ac2a-99d5ff7bdee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:36.436190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:36.480125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-583eed41-2c2d-48bf-ac2a-99d5ff7bdee4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:36.481652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-583eed41-2c2d-48bf-ac2a-99d5ff7bdee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:36.594727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 515/1047] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:36.924949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:36.924949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:36.924949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:36.926869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 533/1048] 10.222.0.39 () {54 vars in 963 bytes} [Wed Oct 19 13:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:36.931133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ac69-11cd-408b-8db1-918f3f9da51b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:36.931427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ac69-11cd-408b-8db1-918f3f9da51b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:36.931702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ac69-11cd-408b-8db1-918f3f9da51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:37.028023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ab44ac69-11cd-408b-8db1-918f3f9da51b None None] MFA Rules not processed for user `bf272634ade542739c5cfe3139bb201a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:37.029243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ab44ac69-11cd-408b-8db1-918f3f9da51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:37.079879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:37.112545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 516/1049] 10.222.0.39 () {58 vars in 1060 bytes} [Wed Oct 19 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:37.120046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:37.121559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:37.174092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:37.230572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8b311-2e54-4ae4-83cf-74e7147249ea None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:37.314267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:37.315427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:37.315708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:37.315991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:37.317654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:37.438652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:37.458440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:37.458440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e381e4-5aee-4fb8-b3eb-ab5135e8545e service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:37.478887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 534/1050] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:38.679097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:38.679703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:38.679703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:38.681016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 517/1051] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:38.686308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e06bf-37f3-4542-9628-b738f4a6d294 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:38.686746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e06bf-37f3-4542-9628-b738f4a6d294 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:38.686954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e06bf-37f3-4542-9628-b738f4a6d294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:38.755027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-db7e06bf-37f3-4542-9628-b738f4a6d294 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:38.756255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-db7e06bf-37f3-4542-9628-b738f4a6d294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:38.857259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 535/1052] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:39.150538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:39.152097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:39.303444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ecec5-676d-4385-9f4c-62087f9cb52e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:39.356137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:39.443118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:39.443118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:39.443118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:39.443118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:39.443653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:39.520388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:39.522032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c36b94-2f12-4b5f-b1ed-427a66f2e0cf service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:39.541257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 518/1053] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:27:41.864026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:41.864603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:41.864603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:41.865928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 536/1054] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:41.871256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708398-72e6-40cd-9e7f-da368043b228 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:41.871674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708398-72e6-40cd-9e7f-da368043b228 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:41.871815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708398-72e6-40cd-9e7f-da368043b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:41.940743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f1708398-72e6-40cd-9e7f-da368043b228 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:41.941962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f1708398-72e6-40cd-9e7f-da368043b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:41.949724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:42.048375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 519/1055] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:42.337273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:42.338979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:42.403637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:42.490638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d51ba7-5750-403a-bd39-b30fecee7d57 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:42.611392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:42.612764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:42.613154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:42.613536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:42.615422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:42.636412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:42.713791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:42.715537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90017525-910a-46e9-af09-f8d7f700936a service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:42.735902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 537/1056] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:45.022121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:45.022735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:45.023379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:45.031248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 520/1057] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:45.032197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa8b8-6636-435d-8fb3-c2c881015433 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:45.033554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa8b8-6636-435d-8fb3-c2c881015433 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:45.033554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fa8b8-6636-435d-8fb3-c2c881015433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:45.099667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1e9fa8b8-6636-435d-8fb3-c2c881015433 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:45.100750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1e9fa8b8-6636-435d-8fb3-c2c881015433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:45.198772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:45.205467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 538/1058] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:45.559121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:45.561164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:45.715910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:45.728816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5d041-ec7d-4345-b69a-14b2a2349aac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:45.854522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:45.855267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:45.855612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:45.855950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:45.857551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:45.959169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:45.960781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290788c-ea8d-424b-b4ed-256a14460f0d service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:45.971202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:45.979342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 521/1059] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:48.291155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:48.291897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:48.291897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:48.292969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 539/1060] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:48.298311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e047cf-663c-4760-964a-f3dfa5bf1649 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:48.298774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e047cf-663c-4760-964a-f3dfa5bf1649 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:48.299063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e047cf-663c-4760-964a-f3dfa5bf1649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:48.371264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-56e047cf-663c-4760-964a-f3dfa5bf1649 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:48.373210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-56e047cf-663c-4760-964a-f3dfa5bf1649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:48.494410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 522/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:48.783747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:48.785275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:48.949377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b65b50-efa5-4060-9da7-b4b98487139e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:49.021692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:49.068874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:49.069429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:49.069706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:49.069974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:49.071692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:49.161320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:49.162936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf1206-1111-4416-899d-3a14bb693ba6 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:49.180848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 540/1062] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:51.314317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:51.314885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:51.314885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:51.316875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 523/1063] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:51.322646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebdf63-e21e-4403-a85a-c732ad402144 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:51.323050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebdf63-e21e-4403-a85a-c732ad402144 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:51.323230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebdf63-e21e-4403-a85a-c732ad402144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:51.389116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-39ebdf63-e21e-4403-a85a-c732ad402144 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:51.390312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-39ebdf63-e21e-4403-a85a-c732ad402144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:51.423507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:51.488817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 541/1064] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:51.777593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:51.779124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:51.854232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:51.923279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83dd6e-4636-4728-a3e5-76b4a4fc7bea None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:52.039178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:52.039777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:52.040067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:52.040337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:52.041956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:52.082411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:52.133998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:52.135650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f84d98-551f-4cef-828c-8c8d373465ae service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:52.154831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 524/1065] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:52.902296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e527ec-4b83-4bd6-8e73-15074cf3d949 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:52.902862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e527ec-4b83-4bd6-8e73-15074cf3d949 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:52.902862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e527ec-4b83-4bd6-8e73-15074cf3d949 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:52.904218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 542/1066] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:52.909461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02f7d6-6a5b-42a0-a3c5-ee0870f2f6dd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:52.910391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02f7d6-6a5b-42a0-a3c5-ee0870f2f6dd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:52.910683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02f7d6-6a5b-42a0-a3c5-ee0870f2f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:52.977736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-df02f7d6-6a5b-42a0-a3c5-ee0870f2f6dd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:52.978975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-df02f7d6-6a5b-42a0-a3c5-ee0870f2f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:53.080327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 525/1067] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:53.386629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:53.386921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:53.387202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:53.388558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 543/1068] 10.222.0.39 () {54 vars in 962 bytes} [Wed Oct 19 13:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:53.393854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55e1e5-6a7a-4761-b027-0b47b5c7ddd8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:53.394245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55e1e5-6a7a-4761-b027-0b47b5c7ddd8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:53.394624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55e1e5-6a7a-4761-b027-0b47b5c7ddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:53.412878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:53.474205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f55e1e5-6a7a-4761-b027-0b47b5c7ddd8 None None] MFA Rules not processed for user `39feed1e4a48413d823bf0ece9b8f59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:53.475526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f55e1e5-6a7a-4761-b027-0b47b5c7ddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:53.570851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 526/1069] 10.222.0.39 () {58 vars in 1059 bytes} [Wed Oct 19 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:53.580603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:53.582334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:53.689780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549530a-170e-437a-bb5f-908d1da60998 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:53.749812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:53.764774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:53.767321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:53.767321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:53.767321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:53.767791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:53.877314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:53.878836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a437b-eeaa-4902-a9d0-bba314446a73 service glance] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:53.897051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 544/1070] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:27:54.473895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614091d-b30e-4f02-b7c9-6de6536ddf23 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:54.474345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614091d-b30e-4f02-b7c9-6de6536ddf23 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:54.474449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614091d-b30e-4f02-b7c9-6de6536ddf23 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:54.475801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 527/1071] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:54.481334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:54.481827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:54.482043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:54.555092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:54.556341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:54.602620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:54.661454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 545/1072] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:54.792434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689276d-2d3a-41ff-b734-34c36ebe5100 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:54.792712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689276d-2d3a-41ff-b734-34c36ebe5100 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:54.792984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689276d-2d3a-41ff-b734-34c36ebe5100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:54.863162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6689276d-2d3a-41ff-b734-34c36ebe5100 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:54.865943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6689276d-2d3a-41ff-b734-34c36ebe5100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:54.920983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:54.953297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:54.954539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:54.956227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 528/1073] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:54.998937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d5f1-7bdf-493d-8824-e50668b46290 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:54.999467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d5f1-7bdf-493d-8824-e50668b46290 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:54.999930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d5f1-7bdf-493d-8824-e50668b46290 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:55.001912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 529/1074] 10.222.0.39 () {54 vars in 943 bytes} [Wed Oct 19 13:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:55.009674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:55.010080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:55.010455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:55.078311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217302ee-835f-4e29-b626-57aba99cff2d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:55.088455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] MFA Rules not processed for user `7498ce3216d1411c961279f321418b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:55.089655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:55.133501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:55.192962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:55.193612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:55.193894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:55.194167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:55.195904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:55.197648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 530/1075] 10.222.0.39 () {58 vars in 1040 bytes} [Wed Oct 19 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:27:55.207403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:55.216697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222bd1d-6504-45e2-8a0f-ec1e0d7f65dc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:55.248686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:55.287370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:55.289031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be4bc5-1e52-46d0-bd4d-de55be98328a service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:55.307284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 546/1076] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:27:55.327825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:55.328431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:55.328710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:55.328982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:55.330657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:55.414752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:55.416323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b88821-3fac-4860-bbe9-52873afefdf5 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:55.424782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 531/1077] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:27:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 13:27:57.657210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:57.657866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:57.657866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:57.659347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 547/1078] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:27:57.665320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05eb3ab-a9ab-4868-92af-7c6b4f876d3f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:57.665854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05eb3ab-a9ab-4868-92af-7c6b4f876d3f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:57.666001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05eb3ab-a9ab-4868-92af-7c6b4f876d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:57.747693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a05eb3ab-a9ab-4868-92af-7c6b4f876d3f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:27:57.748917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a05eb3ab-a9ab-4868-92af-7c6b4f876d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:57.756748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:57.876991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 532/1079] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:27:58.197786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:27:58.199485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:58.277888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:58.366063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5d387-efcc-40a3-b8d1-d3c025480172 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:27:58.495726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:27:58.496427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:27:58.496787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:27:58.497131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:27:58.498980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:27:58.529984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:27:58.603673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:27:58.605283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8a862-5fa0-4632-8c20-fdba1cfc5481 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:27:58.638634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 548/1080] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:01.002157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:01.002157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:01.002157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:01.006841 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 533/1081] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:28:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:01.008670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcc09-2136-4bcd-9598-6e0bfe54f2f7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:01.009117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcc09-2136-4bcd-9598-6e0bfe54f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:01.009230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addcc09-2136-4bcd-9598-6e0bfe54f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:01.080783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9addcc09-2136-4bcd-9598-6e0bfe54f2f7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:01.082038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9addcc09-2136-4bcd-9598-6e0bfe54f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:01.190074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 549/1082] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:01.478025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:01.479645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:01.637715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:01.648671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20060415-315a-44e6-8daf-c2e81566b9f3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:01.776013 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:01.776808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:01.777090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:01.777361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:01.779029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:01.878566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:01.880563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b25b4-50c5-40c0-8825-5558ac5a70b8 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:01.891416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:01.903882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 534/1083] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:04.247358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:04.247884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:04.247995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:04.249274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 550/1084] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:04.255216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0fb33-308e-4d46-9c1d-13fe48c286f3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:04.255980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0fb33-308e-4d46-9c1d-13fe48c286f3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:04.256254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0fb33-308e-4d46-9c1d-13fe48c286f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:04.330495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a3a0fb33-308e-4d46-9c1d-13fe48c286f3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:04.331774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3a0fb33-308e-4d46-9c1d-13fe48c286f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:04.442598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 535/1085] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:28:04.731638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:04.733172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:04.736587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:04.899502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553e155-7dea-4391-a143-3444fe006d0a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:04.990068 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:05.031757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:05.034206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:05.035029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:05.035316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:05.036946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:05.134974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:05.136940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31e90e-b312-4096-9686-8bc1faea03e6 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:05.156558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 551/1086] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:14.438862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=67 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:15.154029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=61 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:19.973449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18778b-2bea-4ae2-a76c-44d4687128f3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:19.973983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18778b-2bea-4ae2-a76c-44d4687128f3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:19.974179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18778b-2bea-4ae2-a76c-44d4687128f3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:19.975653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 536/1087] 10.222.0.39 () {54 vars in 918 bytes} [Wed Oct 19 13:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:19.980918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158a65-bdd7-4420-87b0-c04ec6d86da2 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:19.984153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158a65-bdd7-4420-87b0-c04ec6d86da2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:19.984153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d158a65-bdd7-4420-87b0-c04ec6d86da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:20.058911 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8d158a65-bdd7-4420-87b0-c04ec6d86da2 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:20.060110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8d158a65-bdd7-4420-87b0-c04ec6d86da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:20.163645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 552/1088] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Oct 19 13:28:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:20.178235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fdad2-e0e4-4772-b511-5fb2b2d0e730 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:20.178537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fdad2-e0e4-4772-b511-5fb2b2d0e730 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:20.178852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fdad2-e0e4-4772-b511-5fb2b2d0e730 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:20.180163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 537/1089] 10.222.0.39 () {54 vars in 918 bytes} [Wed Oct 19 13:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:20.184564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8ef2-6a23-4e03-82b4-b1a69e24b4be None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:20.184846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8ef2-6a23-4e03-82b4-b1a69e24b4be None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:20.185115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e8ef2-6a23-4e03-82b4-b1a69e24b4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:20.257067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:20.260004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0e2e8ef2-6a23-4e03-82b4-b1a69e24b4be None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:20.264531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0e2e8ef2-6a23-4e03-82b4-b1a69e24b4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:20.374893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 553/1090] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Oct 19 13:28:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:22.928534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:22.929068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:22.929068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:22.930326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 538/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:22.935517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d909-3f8e-403c-b71a-eb54bc385541 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:22.935955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d909-3f8e-403c-b71a-eb54bc385541 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:22.936064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d909-3f8e-403c-b71a-eb54bc385541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:23.001426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8d56d909-3f8e-403c-b71a-eb54bc385541 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:23.002704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8d56d909-3f8e-403c-b71a-eb54bc385541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:23.057399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:23.099389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 554/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:23.385916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:23.387497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:23.529831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94e85-207c-42ac-92d7-ac0407aa1112 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:23.602468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:23.642347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:23.642916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:23.643194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:23.643462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:23.645036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:23.732557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:23.734159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614566-b6c3-4650-8aae-86ef930cbbb0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:23.752183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 539/1093] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:24.897917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:24.898491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:24.898491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:24.899853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 555/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:24.906226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71cb2-681e-4928-9c6e-76fc99945aca None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:24.906702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71cb2-681e-4928-9c6e-76fc99945aca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:24.906802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71cb2-681e-4928-9c6e-76fc99945aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:24.982744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-53e71cb2-681e-4928-9c6e-76fc99945aca None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:24.983957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-53e71cb2-681e-4928-9c6e-76fc99945aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:25.011775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:25.083297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 540/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:25.243464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:25.244901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:25.388877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b169-ee49-4b97-8632-0453ca6679ac None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:25.428391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:25.499803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:25.500364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:25.500646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:25.500919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:25.503136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:25.592365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:25.593942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8e5f2-0f16-420a-9263-af6dbed99dda service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:25.612986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 556/1096] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:26.737060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:26.737624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:26.737624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:26.738981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 541/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:26.745071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812966b-b4a9-40f9-8a5f-d67507657fab None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:26.745532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812966b-b4a9-40f9-8a5f-d67507657fab None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:26.745649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812966b-b4a9-40f9-8a5f-d67507657fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:26.799649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:26.822229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3812966b-b4a9-40f9-8a5f-d67507657fab None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:26.823399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3812966b-b4a9-40f9-8a5f-d67507657fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:26.917442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 557/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:27.069306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:27.070838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:27.214455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5b6a-7b9b-4d8c-975f-b50602159b9d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:27.226746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:27.325615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:27.326194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:27.326466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:27.326768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:27.328341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:27.416037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:27.417619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec132b9-9f89-4e36-aebb-ee5d14be9037 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:27.435036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 542/1099] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:28.548934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:28.549986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:28.549986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:28.551350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 558/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:28.557460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cebe-1f86-4a77-b9bc-4462b7101904 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:28.557923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cebe-1f86-4a77-b9bc-4462b7101904 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:28.558025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cebe-1f86-4a77-b9bc-4462b7101904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:28.577707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:28.633499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8642cebe-1f86-4a77-b9bc-4462b7101904 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:28.634749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8642cebe-1f86-4a77-b9bc-4462b7101904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:28.730071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 543/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:28.879111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:28.880576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:28.990221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:29.025954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d40f3-e560-465a-9eec-b187f980df8d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:29.136193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:29.136735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:29.137014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:29.137287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:29.138902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:29.215328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:29.229037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:29.238634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eb478-e7f9-4097-b56c-545273c3472e service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:29.248139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 559/1102] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:30.364325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:30.364882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:30.364882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:30.366225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 544/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:30.372475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf312-3351-450b-ab5e-bd529dffdf47 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:30.372885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf312-3351-450b-ab5e-bd529dffdf47 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:30.373135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf312-3351-450b-ab5e-bd529dffdf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:30.448199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ac4bf312-3351-450b-ab5e-bd529dffdf47 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:30.449394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ac4bf312-3351-450b-ab5e-bd529dffdf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:30.544323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 560/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:28:30.696231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:30.697970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:30.781139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:30.842560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41683a6d-d6b0-46d2-80c8-e5a87154ab98 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:30.952950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:30.953478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:30.953758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:30.954030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:30.955694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:31.003827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:31.044318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:31.045935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd394246-c53d-458d-8726-8a230c674133 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:31.064154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 545/1105] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:32.176447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:32.176983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:32.177104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:32.178385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 561/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:32.184596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb4cc7-613b-4417-83c9-cc209dbe8c66 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:32.185035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb4cc7-613b-4417-83c9-cc209dbe8c66 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:32.185165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb4cc7-613b-4417-83c9-cc209dbe8c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:32.261202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-11bb4cc7-613b-4417-83c9-cc209dbe8c66 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:32.262330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-11bb4cc7-613b-4417-83c9-cc209dbe8c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:32.358344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 546/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:32.510058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:32.511718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:32.560291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:32.657380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746bb48-6a6a-4e2c-ba21-5e58935ed8cb None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:32.768364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:32.768888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:32.769163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:32.769429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:32.771034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:32.777750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:32.860173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:32.862295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d50faa-b303-4486-9a6a-61bbff92905b service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:32.880361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 562/1108] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:33.978822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:33.979429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:33.979429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:33.980727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 547/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:33.986977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a009ba-0fbf-4bd7-8aff-57e62b6bebdf None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:33.987427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a009ba-0fbf-4bd7-8aff-57e62b6bebdf None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:33.987567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a009ba-0fbf-4bd7-8aff-57e62b6bebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:34.062614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-18a009ba-0fbf-4bd7-8aff-57e62b6bebdf None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:34.063820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-18a009ba-0fbf-4bd7-8aff-57e62b6bebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:34.131710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:34.159343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 563/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:34.311212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:34.312756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:34.330634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:34.456885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf3e41-2f6f-4e96-8cef-6634904160e0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:34.542346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:34.568079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:34.568615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:34.568893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:34.569163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:34.570827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:34.659650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:34.661243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fb41c-7e10-4a59-b5af-bf8479b58762 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:34.679649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 548/1111] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:35.786257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:35.786854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:35.786854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:35.788163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 564/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:35.794356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1748-f14b-4450-b21d-93c95807851f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:35.794895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1748-f14b-4450-b21d-93c95807851f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:35.795015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1748-f14b-4450-b21d-93c95807851f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:35.871543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e6ff1748-f14b-4450-b21d-93c95807851f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:35.872750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e6ff1748-f14b-4450-b21d-93c95807851f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:35.914003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:35.969471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 549/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:36.117860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:36.119415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:36.263034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f0560-7508-4d11-bc0d-e211eea625ff None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:36.316292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:36.376093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:36.376675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:36.376952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:36.377220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:36.378833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:36.466676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:36.468256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211498dd-d5ce-41c0-b6d9-053b1b4962f4 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:36.486442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 565/1114] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:37.627201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:37.627709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:37.627807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:37.629125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 550/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:37.635347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7f43-04a4-496f-932b-957d653cdaa8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:37.635795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7f43-04a4-496f-932b-957d653cdaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:37.635939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7f43-04a4-496f-932b-957d653cdaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:37.711277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-447c7f43-04a4-496f-932b-957d653cdaa8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:37.712490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-447c7f43-04a4-496f-932b-957d653cdaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:37.714494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:37.809107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 566/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:37.959873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:37.961406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:38.107642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d2bd-f22c-453a-b5d5-4ba3f4d9cb56 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:38.134053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:38.218777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:38.219386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:38.219668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:38.219948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:38.222132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:38.308333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:38.309915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9428-af8d-4537-b88b-422be9bbcc3f service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:38.328026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 551/1117] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:40.406631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:40.407244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:40.407244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:40.408539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 567/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:40.413859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a2a18-6f16-4a8d-a9a9-185f822027d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:40.414287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a2a18-6f16-4a8d-a9a9-185f822027d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:40.414420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a2a18-6f16-4a8d-a9a9-185f822027d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:40.456754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:40.481092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-616a2a18-6f16-4a8d-a9a9-185f822027d1 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:40.482303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-616a2a18-6f16-4a8d-a9a9-185f822027d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:40.578900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 552/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:40.863841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:40.865406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:40.968590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ca606-8e06-41eb-ab38-cf6672787aae None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:41.001781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:41.044678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:41.045294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:41.045575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:41.045849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:41.047471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:41.155349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:41.156696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0210f-45cb-48db-8e16-75bfd7a7a355 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:41.174877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 568/1120] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:43.469282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:43.472651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:43.472651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:43.472651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 553/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:43.476658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd88ed-64bc-4334-8323-d60565103aa9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:43.477064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd88ed-64bc-4334-8323-d60565103aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:43.477248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd88ed-64bc-4334-8323-d60565103aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:43.542916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b8cd88ed-64bc-4334-8323-d60565103aa9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:43.544115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b8cd88ed-64bc-4334-8323-d60565103aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:43.557411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:43.641241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 569/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:28:43.923919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:43.925463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:44.030329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f558-cfbb-4d52-aa36-e01557faf6fa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:44.043553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:44.103393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:44.103990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:44.104263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:44.104530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:44.106092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:44.212452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:44.213810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f637d-e7c2-4b65-84db-ef5ebed5e8e3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:44.232900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 554/1123] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:46.855883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:46.856491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:46.856491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:46.857841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 570/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:46.863617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49324-e74f-453a-91eb-110f7002e693 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:46.864045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49324-e74f-453a-91eb-110f7002e693 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:46.864157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49324-e74f-453a-91eb-110f7002e693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:46.913144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:46.931254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-16d49324-e74f-453a-91eb-110f7002e693 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:46.932508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-16d49324-e74f-453a-91eb-110f7002e693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:47.028901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 555/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:47.315289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:47.316820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:47.417831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d44a42-0401-4148-aacc-598bb0a4b19e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:47.457466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:47.490317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:47.490951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:47.491233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:47.491507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:47.493086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:47.601675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:47.603080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff173-a2bc-4af6-a5c6-7c77cf3eb9aa service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:47.621443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 571/1126] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:49.895449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:49.896021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:49.896021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:49.897254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 556/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:28:49.902662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9fb79-479e-4a1a-9726-c632aba1d230 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:49.903115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9fb79-479e-4a1a-9726-c632aba1d230 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:49.903231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9fb79-479e-4a1a-9726-c632aba1d230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:49.968960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0cd9fb79-479e-4a1a-9726-c632aba1d230 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:49.970193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0cd9fb79-479e-4a1a-9726-c632aba1d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:49.991216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:50.068533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 572/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:50.356437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:50.358010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:50.460308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39e2ba-7c2d-451f-b26f-aa62a2d7b0f1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:50.481402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:50.534621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:50.535216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:50.535494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:50.535763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:50.537388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:50.643742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:50.645108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18a9b-7fc8-4e0a-8569-477a74236275 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:50.663285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 557/1129] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:52.842559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:52.843149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:52.843149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:52.844530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 573/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:52.849969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c45b5-f99a-43bc-ac7c-93d7dd2c5655 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:52.850376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c45b5-f99a-43bc-ac7c-93d7dd2c5655 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:52.850563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c45b5-f99a-43bc-ac7c-93d7dd2c5655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:52.914326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:52.917263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2e2c45b5-f99a-43bc-ac7c-93d7dd2c5655 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:52.918489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2e2c45b5-f99a-43bc-ac7c-93d7dd2c5655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:53.016176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 558/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:53.300418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:53.301950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:53.403724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde13af-2852-4469-862e-aac13adf3bb1 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:53.453527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:53.477257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:53.477851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:53.478129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:53.478395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:53.480007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:53.587989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:53.589416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c4c23-d0ee-4fc7-9877-760c7d45b3db service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:53.607756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 574/1132] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:55.810575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:55.811094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:55.811094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:55.812422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 559/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:28:55.817745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ee6f-88de-4583-9bb2-3f02d4c0a2cb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:55.818181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ee6f-88de-4583-9bb2-3f02d4c0a2cb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:55.818290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743ee6f-88de-4583-9bb2-3f02d4c0a2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:55.884554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c743ee6f-88de-4583-9bb2-3f02d4c0a2cb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:55.885756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c743ee6f-88de-4583-9bb2-3f02d4c0a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:55.915594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:55.983650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 575/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:28:56.269719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:56.271285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:56.372816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884a56-83bb-4d51-91d3-fac9f39b3561 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:56.401653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:56.446977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:56.447578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:56.447861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:56.448139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:56.449729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:56.560424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:56.566665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bafd70-fef0-43e8-b881-0f81df366d1a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:56.581293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 560/1135] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:28:58.778129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:58.778720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:58.778720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:58.780230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 576/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:28:58.786308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58721-3f34-4778-b53d-3e86e6e14330 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:58.786803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58721-3f34-4778-b53d-3e86e6e14330 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:58.786911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58721-3f34-4778-b53d-3e86e6e14330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:58.853847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b7f58721-3f34-4778-b53d-3e86e6e14330 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:28:58.855090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b7f58721-3f34-4778-b53d-3e86e6e14330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:58.864004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:58.950890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 561/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:28:59.234171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:28:59.235805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:59.338137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3878a-388c-4cf8-a02f-ac6b08f451f1 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:28:59.393363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:28:59.411048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:28:59.411907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:28:59.412185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:28:59.412457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:28:59.414018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:28:59.522353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:28:59.523749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ed504-ea70-41a6-80e7-3822211b2496 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:28:59.541641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 577/1138] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:01.742899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:01.743457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:01.743457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:01.744812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 562/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:01.750145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a621c-12fc-4c79-9190-45b93c7bf4ef None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:01.750580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a621c-12fc-4c79-9190-45b93c7bf4ef None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:01.750743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a621c-12fc-4c79-9190-45b93c7bf4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:01.816047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-593a621c-12fc-4c79-9190-45b93c7bf4ef None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:01.817160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-593a621c-12fc-4c79-9190-45b93c7bf4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:01.855542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:01.913066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 578/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:02.199707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:02.201242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:02.302375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d372b-bec7-45f8-9121-1378d2f69a4a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:02.340907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:02.376994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:02.377595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:02.377872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:02.378140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:02.379749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:02.486353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:02.487735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199727dc-61f7-4b8a-8ffa-43145381a10b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:02.506347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 563/1141] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:04.702594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:04.703139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:04.703139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:04.705119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 579/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:04.710388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44fca3-1392-4092-a23b-162d907c0b78 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:04.710843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44fca3-1392-4092-a23b-162d907c0b78 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:04.711039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44fca3-1392-4092-a23b-162d907c0b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:04.777454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-be44fca3-1392-4092-a23b-162d907c0b78 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:04.778697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be44fca3-1392-4092-a23b-162d907c0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:04.796433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:04.873347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 564/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:29:05.153841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:05.155396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:05.258315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f69e-6628-4468-9e6e-89c8583980b2 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:05.324935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:05.331378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:05.331980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:05.332262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:05.332535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:05.334118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:05.443486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:05.444909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8928c-a801-48f1-906a-0108a5941001 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:05.462683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 580/1144] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:29:07.608596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:07.609093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:07.609187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:07.611183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 565/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:07.616435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae732-2a09-493c-a1c5-83b1a956d37d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:07.616869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae732-2a09-493c-a1c5-83b1a956d37d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:07.616968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae732-2a09-493c-a1c5-83b1a956d37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:07.681962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c86ae732-2a09-493c-a1c5-83b1a956d37d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:07.683174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c86ae732-2a09-493c-a1c5-83b1a956d37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:07.730566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:07.778161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 581/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:08.064398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:08.065944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:08.170201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848999-33b0-4339-82ca-441eb64ab1b2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:08.216878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:08.242803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:08.243516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:08.243790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:08.244054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:08.245633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:08.352361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:08.353789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376e8af-6da4-4cfa-9724-827ef6558b8a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:08.371826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 566/1147] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:17.776619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=22 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:18.369506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=66 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:28.326317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a082a-2d06-489d-bc36-c327fec08a16 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:28.326922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a082a-2d06-489d-bc36-c327fec08a16 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:28.326922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a082a-2d06-489d-bc36-c327fec08a16 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:28.328291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 582/1148] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:28.333770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bed6-26d0-4b55-bb8e-ae44e0c1dd39 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:28.334172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bed6-26d0-4b55-bb8e-ae44e0c1dd39 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:28.334331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0bed6-26d0-4b55-bb8e-ae44e0c1dd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:28.400422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c6e0bed6-26d0-4b55-bb8e-ae44e0c1dd39 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:28.401614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c6e0bed6-26d0-4b55-bb8e-ae44e0c1dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:28.502641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 567/1149] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:28.802488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:28.802801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:28.803073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:28.804395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 583/1150] 10.222.0.39 () {54 vars in 962 bytes} [Wed Oct 19 13:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:28.809241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed89c-97ae-4ea3-8864-0fdb0286cd14 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:28.809535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed89c-97ae-4ea3-8864-0fdb0286cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:28.809817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ed89c-97ae-4ea3-8864-0fdb0286cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:28.885333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e98ed89c-97ae-4ea3-8864-0fdb0286cd14 None None] MFA Rules not processed for user `39feed1e4a48413d823bf0ece9b8f59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:28.886565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e98ed89c-97ae-4ea3-8864-0fdb0286cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:28.907772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:28.962347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 568/1151] 10.222.0.39 () {58 vars in 1059 bytes} [Wed Oct 19 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:28.970728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:28.972297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:29.074248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00242a-26ad-4cfe-8c58-b2e0e61f08b4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:29.145938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:29.146573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:29.146853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:29.147118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:29.148690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:29.201354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:29.256181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:29.257671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f61ae2-099f-4a12-8b61-7ba854245ac7 service glance] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:29.277731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 584/1152] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:31.500148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968657-1913-4587-9169-d7e4be187302 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:31.500736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968657-1913-4587-9169-d7e4be187302 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:31.500736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9968657-1913-4587-9169-d7e4be187302 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:31.502042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 569/1153] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:31.507362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac48363-05f3-495c-a030-c97be535140e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:31.507826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac48363-05f3-495c-a030-c97be535140e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:31.508004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac48363-05f3-495c-a030-c97be535140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:31.572958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7ac48363-05f3-495c-a030-c97be535140e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:31.574152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ac48363-05f3-495c-a030-c97be535140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:31.668504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 585/1154] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:31.943812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9968657-1913-4587-9169-d7e4be187302 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:31.945616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a9968657-1913-4587-9169-d7e4be187302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:32.068750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9968657-1913-4587-9169-d7e4be187302 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:32.118130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:32.161777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:32.162374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:32.162693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:32.162967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:32.163603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:32.164694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:29:32.164694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:29:32.164694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:29:32.165839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd71e-5b44-46f4-abd6-5789a1a71435 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:32.185233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 570/1155] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Oct 19 13:29:31 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:29:34.347805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:34.348360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:34.348360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:34.349692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 586/1156] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:34.354881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa051f-18c2-44fd-96d7-3e70d93afca8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:34.355287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa051f-18c2-44fd-96d7-3e70d93afca8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:34.355438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa051f-18c2-44fd-96d7-3e70d93afca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:34.421420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9efa051f-18c2-44fd-96d7-3e70d93afca8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:34.422683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9efa051f-18c2-44fd-96d7-3e70d93afca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:34.517326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 571/1157] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:34.807172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:34.808683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:34.818988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:34.955540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f8a1d-3615-421d-9bf1-69ff147a3c47 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:35.034347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:35.067425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:35.068038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:35.068325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:35.068601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:35.070521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:35.161099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:35.162728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bac434-e8e3-49aa-8b60-8521325c84f5 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:35.180321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 587/1158] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:37.480877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:37.481446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:37.481446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:37.482849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 572/1159] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:37.488591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022847e-fb3c-413d-b9cb-5801c7492e7b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:37.489013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022847e-fb3c-413d-b9cb-5801c7492e7b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:37.489122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022847e-fb3c-413d-b9cb-5801c7492e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:37.553752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1022847e-fb3c-413d-b9cb-5801c7492e7b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:37.554960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1022847e-fb3c-413d-b9cb-5801c7492e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:37.599830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:37.648301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 588/1160] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:29:37.932333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:37.933959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:37.958182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:38.037476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585760dc-a9d7-4ecf-98d4-457c05d4fbd5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:38.109627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:38.110226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:38.110500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:38.110798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:38.112370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:38.184893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:38.220233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:38.221660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a738d16-db9b-4384-9068-972eac159a3b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:38.239788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 573/1161] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:40.443159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:40.443733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:40.443733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:40.445289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 589/1162] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:40.450743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195f36-3634-4c06-ad47-b3400bfae007 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:40.451188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195f36-3634-4c06-ad47-b3400bfae007 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:40.451309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90195f36-3634-4c06-ad47-b3400bfae007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:40.516966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90195f36-3634-4c06-ad47-b3400bfae007 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:40.518146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90195f36-3634-4c06-ad47-b3400bfae007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:40.613358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 574/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:40.900233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:40.901784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:41.004929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0ba94-e1ea-4648-9d92-b9f2c6607b66 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:41.076771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:41.078765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:41.079371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:41.079651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:41.079923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:41.081518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:41.188165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:41.189517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795f67-8c53-434e-bcc6-0e5b0fc63eeb service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:41.207766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 590/1164] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:43.428824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:43.429433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:43.429433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:43.430764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 575/1165] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:43.435889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625b484-1f2d-40a9-b025-ad1e1485f0d9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:43.436299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625b484-1f2d-40a9-b025-ad1e1485f0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:43.436447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625b484-1f2d-40a9-b025-ad1e1485f0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:43.502671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d625b484-1f2d-40a9-b025-ad1e1485f0d9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:43.503858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d625b484-1f2d-40a9-b025-ad1e1485f0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:43.555075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:43.598731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 591/1166] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:43.889609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:43.891663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:43.927425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:43.995004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7bac-f58f-4533-b40e-f73bd1c5c9cc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:44.068652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:44.069256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:44.069535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:44.070062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:44.071666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:44.150646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:44.179344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:44.180811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa96039-bcaf-4df8-8e92-92d234bc739e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:44.200071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 576/1167] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:46.831113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:46.831961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:46.832137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:46.833672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 592/1168] 10.222.0.39 () {54 vars in 927 bytes} [Wed Oct 19 13:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:29:46.838809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd841bb-dc26-4647-9861-56dc909f7e8b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:46.839247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd841bb-dc26-4647-9861-56dc909f7e8b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:46.839391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd841bb-dc26-4647-9861-56dc909f7e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:46.904494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4fd841bb-dc26-4647-9861-56dc909f7e8b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:29:46.905715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4fd841bb-dc26-4647-9861-56dc909f7e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:47.002838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 577/1169] 10.222.0.39 () {58 vars in 1024 bytes} [Wed Oct 19 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:29:47.289560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:29:47.291126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:47.393260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dbac8-3f01-4482-a5a0-69f21e12e09e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:29:47.465220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:29:47.465816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:29:47.466094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:29:47.466368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:29:47.468476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:29:47.476742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:47.576225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:29:47.577653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638c104-fa1d-40f1-b0a5-e428382fc902 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:29:47.595901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 593/1170] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:29:57.001102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:29:57.593132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=50 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:10.984742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:10.985352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:10.985352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:11.060757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:31:11.061980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:11.157612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 578/1171] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:31:11.165485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:11.165957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:11.166066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:11.242774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:31:11.243889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:11.337809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 594/1172] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:31:11.343018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:11.344223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:11.355341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0983917-89f9-4bd6-b65d-9d73431b6a2a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:11.417237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:11.449064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:11.449666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:11.449945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:11.450218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:11.450888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:31:11.451192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:11.452692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:31:11.452692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:31:11.452692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:31:11.453584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:11.465488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 579/1173] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:31:11.475513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:11.484690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9946a-86c4-4875-8e57-0690a32164c0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:11.545180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:11.578430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:11.579054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:11.579330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:11.579597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:11.580231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:11.581325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:31:11.581325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:31:11.581325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:31:11.582501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:11.684762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 595/1174] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:31:11 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:31:11.689477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:11.691072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:11.724061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b489-e983-49ad-8197-6cc8815165cc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:11.815939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:11.816536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:11.816811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:11.817076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:11.817969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:11.818999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:31:11.818999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:31:11.818999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:31:11.820186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:11.940944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 580/1175] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:31:11 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:31:11.945386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:11.946988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.008951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08876d-d464-4d8a-8720-c0e4c5c687d2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:12.091975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:12.101865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:12.102445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.102757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.103033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.103674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:12.105272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:12.118561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 596/1176] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:31:12.122578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:12.124787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.135992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97814-a172-45b8-8687-50139591f8cf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:12.160828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:12.230148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:12.230775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.231052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.231320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.231951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:12.234000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7919de1-2d65-477b-9725-5cdca84c355a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:12.246582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 581/1177] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:31:12.251316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:12.252829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.285705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436dc199-6a60-4d68-a5df-87e155ca72aa admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:12.377089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:12.377698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.377975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.378244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] PATH_INFO: `/v3/projects/6bd6115c1c144f71a9ba65d6d0c8abcd/users/31ea725f96e2415c8ff9b8b63a01b7c6/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.399652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd6115c1c144f71a9ba65d6d0c8abcd, user_id=31ea725f96e2415c8ff9b8b63a01b7c6, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:12.402271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e534b45-b889-4396-b10e-e937963cf357 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:12.475129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 597/1178] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:31:12 2022] PUT /identity/v3/projects/6bd6115c1c144f71a9ba65d6d0c8abcd/users/31ea725f96e2415c8ff9b8b63a01b7c6/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:31:12.486867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66ba1-7f23-44e9-833c-049ea7253430 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.487176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66ba1-7f23-44e9-833c-049ea7253430 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.487436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc66ba1-7f23-44e9-833c-049ea7253430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.543020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7fc66ba1-7f23-44e9-833c-049ea7253430 None None] MFA Rules not processed for user `31ea725f96e2415c8ff9b8b63a01b7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:31:12.544227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7fc66ba1-7f23-44e9-833c-049ea7253430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.622256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 582/1179] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:31:12.635541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.635836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.636116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.637440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 598/1180] 10.222.0.39 () {54 vars in 963 bytes} [Wed Oct 19 13:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:31:12.642213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244c1e7-ad83-4de5-a89f-b1bc0e7628d0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.642535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244c1e7-ad83-4de5-a89f-b1bc0e7628d0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.642821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244c1e7-ad83-4de5-a89f-b1bc0e7628d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.674569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:12.719608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d244c1e7-ad83-4de5-a89f-b1bc0e7628d0 None None] MFA Rules not processed for user `bf272634ade542739c5cfe3139bb201a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:31:12.720814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d244c1e7-ad83-4de5-a89f-b1bc0e7628d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.795549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 583/1181] 10.222.0.39 () {58 vars in 1060 bytes} [Wed Oct 19 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:31:12.803284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:12.804850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:12.854889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:12.908274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405f351-cb59-44e2-864d-98178059c9f4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:12.981287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:12.981898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:12.982182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:12.982456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:12.984099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:13.081693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:13.083167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:13.086637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:13.102775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 599/1182] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:31:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:31:13.148114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9dc4d-f71f-480a-bbd0-33d3518cbb76 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:13.148538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9dc4d-f71f-480a-bbd0-33d3518cbb76 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:13.148661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9dc4d-f71f-480a-bbd0-33d3518cbb76 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:13.149904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 584/1183] 10.222.0.39 () {52 vars in 830 bytes} [Wed Oct 19 13:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:31:13.153324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:13.154576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:13.251716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f1583-1bbe-4b9f-ab97-94f731a522a7 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:13.343713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:13.344339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:13.344620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:13.344894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] PATH_INFO: `/v3/users/31ea725f96e2415c8ff9b8b63a01b7c6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:13.357336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ea725f96e2415c8ff9b8b63a01b7c6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:13.358394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:31:13.358394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:31:13.358394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:31:13.359637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:13.369155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:13.578828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ea725f96e2415c8ff9b8b63a01b7c6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:13.592696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ea725f96e2415c8ff9b8b63a01b7c6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:13.615753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ea725f96e2415c8ff9b8b63a01b7c6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:13.616207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ea725f96e2415c8ff9b8b63a01b7c6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:13.677226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ea725f96e2415c8ff9b8b63a01b7c6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:13.683533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 600/1184] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:31:13 2022] DELETE /identity/v3/users/31ea725f96e2415c8ff9b8b63a01b7c6 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:31:13.688548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:13.688835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:13.689112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:13.745275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:31:13.746478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:13.820882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:13.833141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 585/1185] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:31:13.844115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:13.848671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:13.952612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9cd2a-7fa7-4e70-855b-8057bb93ba1b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:14.008755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:14.027780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:14.028385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:14.028663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:14.028935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:14.031085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:14.140733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:14.142095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a92c0-ea6d-4a80-a171-c34984ae6ffa service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:14.160633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 601/1186] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:31:14.619197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:31:14.623182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:31:14.632893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ea68-fbb9-4022-b2e1-3839f4619d17 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:31:14.732865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:31:14.733518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:31:14.733799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:31:14.734069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] PATH_INFO: `/v3/projects/6bd6115c1c144f71a9ba65d6d0c8abcd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:31:14.744434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd6115c1c144f71a9ba65d6d0c8abcd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:31:14.745458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:31:14.745458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:31:14.745458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:31:14.746703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:31:14.850618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd6115c1c144f71a9ba65d6d0c8abcd', 'request_id': 'req-fd5828aa-bfed-45bd-b96a-2204ec414362'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:14.856667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd6115c1c144f71a9ba65d6d0c8abcd', 'request_id': 'req-fd5828aa-bfed-45bd-b96a-2204ec414362'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:14.922829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd6115c1c144f71a9ba65d6d0c8abcd', 'request_id': 'req-fd5828aa-bfed-45bd-b96a-2204ec414362'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:14.954316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] The token cache is being invalidate because project 6bd6115c1c144f71a9ba65d6d0c8abcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:31:14.954634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:14.955066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd5828aa-bfed-45bd-b96a-2204ec414362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:31:14.956453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 586/1187] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:31:14 2022] DELETE /identity/v3/projects/6bd6115c1c144f71a9ba65d6d0c8abcd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:31:24.158389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=62 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:31:24.951722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:32:44.469727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:44.470351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:44.470351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:44.471653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 602/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:32:44.476583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb0e-627b-4bf0-9c76-3f26c2c80280 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:44.476996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb0e-627b-4bf0-9c76-3f26c2c80280 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:44.477136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7cb0e-627b-4bf0-9c76-3f26c2c80280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:44.559059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-67e7cb0e-627b-4bf0-9c76-3f26c2c80280 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:44.560260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67e7cb0e-627b-4bf0-9c76-3f26c2c80280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:44.662379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 587/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:32:44.816802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:32:44.819175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:44.938300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f20ec5-2cef-4741-bacb-6db4aeb539c6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:32:44.944036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:32:44.948915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:44.948915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:44.948915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:44.953107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:32:44.955275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6027b-52c9-488a-aa5f-ea1b3901924e service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:32:44.964883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 603/1190] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:32:46.046909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183c20e-3202-45b1-b674-1d39861e6447 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:46.048098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183c20e-3202-45b1-b674-1d39861e6447 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:46.048098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183c20e-3202-45b1-b674-1d39861e6447 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:46.049442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 588/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:32:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:32:46.056168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b132-ce95-4cb2-a940-7d1a94cb6c7a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:46.056632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b132-ce95-4cb2-a940-7d1a94cb6c7a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:46.056744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b132-ce95-4cb2-a940-7d1a94cb6c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:46.076313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0050b132-ce95-4cb2-a940-7d1a94cb6c7a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:46.077506 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0050b132-ce95-4cb2-a940-7d1a94cb6c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:46.085365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 604/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:32:46.239374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183c20e-3202-45b1-b674-1d39861e6447 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:32:46.253549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183c20e-3202-45b1-b674-1d39861e6447 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:32:46.259058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:32:46.259602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:46.259882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:46.260155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:46.266978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f9cb-fd27-4540-9859-cee74dd97e0a None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:46.267274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f9cb-fd27-4540-9859-cee74dd97e0a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:46.267558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f9cb-fd27-4540-9859-cee74dd97e0a None None] PATH_INFO: `/v3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:46.268285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:32:46.268980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 605/1193] 10.222.0.23 () {54 vars in 925 bytes} [Wed Oct 19 13:32:46 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:32:46.269898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cd361-90a7-49ce-868a-a51b9b79248c service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:32:46.275043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d3e93-7b52-4831-a25e-9b6831c62a4a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:46.275331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d3e93-7b52-4831-a25e-9b6831c62a4a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:46.275608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d3e93-7b52-4831-a25e-9b6831c62a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:46.279493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 589/1194] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:32:46.344909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-019d3e93-7b52-4831-a25e-9b6831c62a4a None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:46.346146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-019d3e93-7b52-4831-a25e-9b6831c62a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:46.403110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:32:46.422368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 606/1195] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 19 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:32:55.471322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f70-46d7-4da4-852a-2258d9cdf5a5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:55.471807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f70-46d7-4da4-852a-2258d9cdf5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:55.471918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f70-46d7-4da4-852a-2258d9cdf5a5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:55.473372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 590/1196] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:32:55.478436 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e445373-031b-42ba-8a08-a8b770b450af None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:55.478889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e445373-031b-42ba-8a08-a8b770b450af None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:55.479029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e445373-031b-42ba-8a08-a8b770b450af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:55.499151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9e445373-031b-42ba-8a08-a8b770b450af None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:55.500967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9e445373-031b-42ba-8a08-a8b770b450af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:55.509429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 607/1197] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:32:55.516730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872f70-46d7-4da4-852a-2258d9cdf5a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:32:55.529813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872f70-46d7-4da4-852a-2258d9cdf5a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:32:55.534641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:32:55.535230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:55.535510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:55.535784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:55.544107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:32:55.545694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c4785-29d9-4a15-8067-984c7fe84c4d service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:32:55.555771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 591/1198] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:32:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:32:57.758014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c4b4-ae5a-4169-a103-40bff5ecc4c9 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:57.758491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c4b4-ae5a-4169-a103-40bff5ecc4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:57.758751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1c4b4-ae5a-4169-a103-40bff5ecc4c9 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:57.759969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 608/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:32:57.766600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8bdb-d8ed-4814-84de-fb6c06e2480b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:57.767014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8bdb-d8ed-4814-84de-fb6c06e2480b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:57.767146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8bdb-d8ed-4814-84de-fb6c06e2480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:57.786946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6dab8bdb-d8ed-4814-84de-fb6c06e2480b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:57.788140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6dab8bdb-d8ed-4814-84de-fb6c06e2480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:57.795793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 592/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:32:57.948490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1c4b4-ae5a-4169-a103-40bff5ecc4c9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:32:57.963845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1c4b4-ae5a-4169-a103-40bff5ecc4c9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:32:57.970007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:32:57.970651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:57.970940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:57.971218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:57.979442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:32:57.986260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c45a-9a52-4ef8-8015-7c6f398da054 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:32:57.990561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 609/1201] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:32:59.053944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8f28-541d-4d9b-a9c7-a0e2834bcd6b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:59.054475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8f28-541d-4d9b-a9c7-a0e2834bcd6b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:59.054475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f8f28-541d-4d9b-a9c7-a0e2834bcd6b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:59.055865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 593/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 13:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:32:59.061878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6513a-11c1-441d-9d57-dd175aef4acb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:59.062357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6513a-11c1-441d-9d57-dd175aef4acb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:59.062494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6513a-11c1-441d-9d57-dd175aef4acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:59.083127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ffc6513a-11c1-441d-9d57-dd175aef4acb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:32:59.084330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ffc6513a-11c1-441d-9d57-dd175aef4acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:32:59.092189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 610/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 13:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:32:59.244149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f8f28-541d-4d9b-a9c7-a0e2834bcd6b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:32:59.258329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f8f28-541d-4d9b-a9c7-a0e2834bcd6b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:32:59.263649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:32:59.264245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:32:59.264522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:32:59.264794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:32:59.273276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:32:59.274916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b1ba3-3087-4198-9757-7d713934a0c4 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:32:59.285247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 594/1204] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:33:03.230004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:03.230606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:03.230606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:03.231971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 611/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:33:03.237075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcc4-ed42-4783-954b-efd72f5d5f49 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:03.237522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcc4-ed42-4783-954b-efd72f5d5f49 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:03.237632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcc4-ed42-4783-954b-efd72f5d5f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:03.257106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3295dcc4-ed42-4783-954b-efd72f5d5f49 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:33:03.258210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3295dcc4-ed42-4783-954b-efd72f5d5f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:03.266890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 595/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:33:03.552975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:33:03.554746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:03.633666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfb58-b8bc-42ed-935e-1b69a9fa3b0e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:33:03.637270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:33:03.637870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:03.638147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:03.638419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:03.647273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:33:03.648686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd81eb-46c0-44c9-ad0c-831e85ec1fa2 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:33:03.658202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 612/1207] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:33:05.789271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790520-bb38-40e0-b164-629969c9fabf None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:05.789835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790520-bb38-40e0-b164-629969c9fabf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:05.789835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca790520-bb38-40e0-b164-629969c9fabf None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:05.791202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 596/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:33:05.797016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094bb98-e337-4dcf-8fb6-32d69d9f8b68 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:05.797442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094bb98-e337-4dcf-8fb6-32d69d9f8b68 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:05.797564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094bb98-e337-4dcf-8fb6-32d69d9f8b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:05.816928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c094bb98-e337-4dcf-8fb6-32d69d9f8b68 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:33:05.818106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c094bb98-e337-4dcf-8fb6-32d69d9f8b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:05.825993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 613/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:33:06.110280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca790520-bb38-40e0-b164-629969c9fabf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:33:06.115673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:33:06.121329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca790520-bb38-40e0-b164-629969c9fabf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:33:06.124900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:33:06.125522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:06.126032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:06.126127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:06.134784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:33:06.136205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd03fb0-e02e-4a2d-8ea6-f7d35c5ac2e1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:33:06.145815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 597/1210] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:33:09.736395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:09.736859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:09.736953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:09.738273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 614/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:33:09.743460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfd2d5-96cf-4423-a960-5ac6627df10f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:09.743904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfd2d5-96cf-4423-a960-5ac6627df10f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:09.744016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfd2d5-96cf-4423-a960-5ac6627df10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:09.764479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6cfd2d5-96cf-4423-a960-5ac6627df10f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:33:09.766177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6cfd2d5-96cf-4423-a960-5ac6627df10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:09.774072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 598/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:33:10.079043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:33:10.080711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:10.158634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f33d-d6b1-4176-ba42-19cbb68f0461 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:33:10.162185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:33:10.162818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:10.163099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:10.163372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:10.172092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:33:10.173516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f971158-a214-4c79-94d8-b59a57dcb304 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:33:10.182720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 615/1213] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:33:12.306375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:12.306922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:12.307044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:12.308301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 599/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 13:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:33:12.313641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eff15-b7c2-4a69-8e51-20e824dc1a77 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:12.314052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eff15-b7c2-4a69-8e51-20e824dc1a77 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:12.314201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eff15-b7c2-4a69-8e51-20e824dc1a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:12.333747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b56eff15-b7c2-4a69-8e51-20e824dc1a77 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:33:12.334977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b56eff15-b7c2-4a69-8e51-20e824dc1a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:12.343555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 616/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:33:12.642478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:33:12.644217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:33:12.657522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557f58-6d78-44ff-89b1-f11c8f6e19fb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:33:12.661046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:33:12.661920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:33:12.661920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:33:12.662179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:33:12.670980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:33:12.672405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6e749-64ae-44c5-ae8a-4b3830f42e39 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:33:12.681671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 600/1216] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:33:22.328896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:33:22.677569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=14 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:37.921007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87898-8fb1-4dab-ae5d-ae1af2859318 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:37.924652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87898-8fb1-4dab-ae5d-ae1af2859318 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:37.924652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be87898-8fb1-4dab-ae5d-ae1af2859318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:37.951558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5be87898-8fb1-4dab-ae5d-ae1af2859318 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:37.953196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be23cc-049f-4cc4-b759-4c586c8058d7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:37.953734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5be87898-8fb1-4dab-ae5d-ae1af2859318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:37.954286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be23cc-049f-4cc4-b759-4c586c8058d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:37.954815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be23cc-049f-4cc4-b759-4c586c8058d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:37.971245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 617/1217] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:37.979382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:37.979989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:37.980539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:37.988881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-56be23cc-049f-4cc4-b759-4c586c8058d7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:37.990099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-56be23cc-049f-4cc4-b759-4c586c8058d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.008194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 601/1218] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:38.020817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.022163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.024183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.024483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.024764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.038396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 618/1219] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:38.043301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.057912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.059185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.060567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30153694-089f-4b86-b150-0497cffc1d22 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.067390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.068346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.068788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.069219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.070038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:38.070523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.072383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:34:38.072383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:34:38.072383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:34:38.072680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 602/1220] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.073807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.079936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.093799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 619/1221] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:34:38.096650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67464f3-d6ce-4bfd-a275-f6ddfdb606d9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.102554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.109094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.110164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.110788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.111307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.112230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:38.112774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.114695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.122706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cba6-64be-43fb-8c45-5edbe981f2ea admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.133074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.133074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.133074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.133472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.134077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.138532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.142726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 603/1222] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:34:38.151613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.165192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f62d9-859d-49c1-b156-06a8306dcf6c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.192062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.194575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.194575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.194575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.194575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.195885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:34:38.195885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:34:38.195885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:34:38.197641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.312347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 620/1223] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:34:38.317150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.351519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 604/1224] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:38.356056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.410804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61533c-349c-4e6d-918d-aec470ea42a2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.511798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.512440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.512722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.512998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.513655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.514763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:34:38.514763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:34:38.514763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:34:38.516051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b886206-23a1-484f-bcb5-766a061aeb19 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.527593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94230dad-3f5c-4ea4-b089-59d8f9ec61f4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.547280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.548337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.548846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.549347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.550239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.552804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33298f62-6aec-4ed9-9458-97792cbe7533 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.641885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 621/1225] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/users => generated 292 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:34:38.643247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1629bc-8fda-43f8-80f0-4533a11fe10e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.643596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1629bc-8fda-43f8-80f0-4533a11fe10e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.643878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1629bc-8fda-43f8-80f0-4533a11fe10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.670869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2e1629bc-8fda-43f8-80f0-4533a11fe10e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.672313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2e1629bc-8fda-43f8-80f0-4533a11fe10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.682520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 622/1226] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.684018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.684303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.684575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.689432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 605/1227] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/users => generated 295 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:34:38.691153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.691631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.692116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.708671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.709913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.721183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 623/1228] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.722116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.733886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb92fcc-db1d-4643-802a-2ea06c5eaff7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.739367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.740166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.740515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.740794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.741491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.743381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4ca5e-1169-44a7-a932-0a39b9461d83 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.743589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.745078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.755160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 624/1229] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:38.756457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab94f6-ed87-42ed-a107-6becac3f62eb None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.756744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab94f6-ed87-42ed-a107-6becac3f62eb None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.757018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab94f6-ed87-42ed-a107-6becac3f62eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.765126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 606/1230] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.765974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.784266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4558af9-8261-4392-9146-041cba6335c9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.789698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.790313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.790576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-87ab94f6-ed87-42ed-a107-6becac3f62eb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.790719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.790956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.791688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.792065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-87ab94f6-ed87-42ed-a107-6becac3f62eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.793278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5225-a16d-4241-8818-e80f96779178 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.805707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 625/1231] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.805707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750701c0-ea49-4070-b805-d62cfbd92870 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.805707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750701c0-ea49-4070-b805-d62cfbd92870 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.805707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750701c0-ea49-4070-b805-d62cfbd92870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.807831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 607/1232] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:38.809113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.809403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.809681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.824019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-750701c0-ea49-4070-b805-d62cfbd92870 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.825212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-750701c0-ea49-4070-b805-d62cfbd92870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.830595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.831790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.833649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 626/1233] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.834862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3e9f7-9032-4849-8f3e-9b37c1c5aab9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.835144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3e9f7-9032-4849-8f3e-9b37c1c5aab9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.835417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3e9f7-9032-4849-8f3e-9b37c1c5aab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.840405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 608/1234] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.842283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.844345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:38.852477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16915a9-05b1-4640-8a33-851a98ed25ac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.856851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4af3e9f7-9032-4849-8f3e-9b37c1c5aab9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.857067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.857669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.857979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.858126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4af3e9f7-9032-4849-8f3e-9b37c1c5aab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.858260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.858936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.860541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.866272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 627/1235] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.867524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.867804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.868080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.880586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 609/1236] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:38.881428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.889814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.891057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.891956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322155a5-0244-4ae6-8873-5fa74aabcaec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.896396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.897012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.897292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.897560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.898198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:38.898485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.900124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c7484-da04-4dd1-b33a-5ab7cf7bccbe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.903803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 628/1237] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.903803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.911908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919f108-b53c-4ec3-9d6f-bdb5ab66de8c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.912655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 610/1238] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:38.913706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5fba7-0df6-4732-bf52-4d9ed05f7c27 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.913987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5fba7-0df6-4732-bf52-4d9ed05f7c27 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.914257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5fba7-0df6-4732-bf52-4d9ed05f7c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.923712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.924339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.924626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.924959 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.925722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.927367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.933756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-07f5fba7-0df6-4732-bf52-4d9ed05f7c27 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.935025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-07f5fba7-0df6-4732-bf52-4d9ed05f7c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.942170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 629/1239] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:38.942170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.943808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 611/1240] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.945039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.945325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.945640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.949454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceccfd-60be-411f-9027-9bb4809609f9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.955829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.956511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.956986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.957451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.958280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:38.958933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:38.961042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d5ba7-8b40-41d8-8723-f60395e23314 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:38.966022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.974044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:38.974490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 630/1241] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:38.975841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.976131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.976246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 612/1242] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:38.976764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.977143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:38.989779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502f15d-49c7-446b-91cf-3e4c487fc2f4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:38.994661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:38.995313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:38.995594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:38.995913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] PATH_INFO: `/v3/projects/ddae0cb1d13b4d4bbbcb2ba46a39f6d0/users/fd3620e3535b4040896736ce79e38fd7/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:38.997013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:38.998214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:39.007241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 631/1243] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:39.008021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.018273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddae0cb1d13b4d4bbbcb2ba46a39f6d0, user_id=fd3620e3535b4040896736ce79e38fd7, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.018476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e6a21-faca-4847-a873-b7bcfa494c87 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.021154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.023096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.023722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.024010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.024305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.024953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:39.025357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.026979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.038790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 632/1244] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:39.039553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.048651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc683d-a945-4f22-8b89-fba7a95f09f1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.053125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.053688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.053973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.054257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.054905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.056751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3301be9-d590-4738-90b3-d28e69533af7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.088253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 613/1245] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:38 2022] PUT /identity/v3/projects/ddae0cb1d13b4d4bbbcb2ba46a39f6d0/users/fd3620e3535b4040896736ce79e38fd7/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:39.089269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.106957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:39.137133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 633/1246] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.138630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.138922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.139902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.151780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b84f0d-bb97-4bd4-83ac-436612104167 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.160106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:39.161348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:39.211171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.211838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.212149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.212388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] PATH_INFO: `/v3/projects/bc8fa61ce6ff491b9d497db42d900f50/users/6c828726ae814c0f879a5aa69f647a96/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.220990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 634/1247] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:39.221815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.229764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8fa61ce6ff491b9d497db42d900f50, user_id=6c828726ae814c0f879a5aa69f647a96, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.230971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de42e4-bd2d-4e0b-85ec-ffe38640ca36 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.232507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.236263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.236863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.237145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.237426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.238089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.240030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.284462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 614/1248] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:39 2022] PUT /identity/v3/projects/bc8fa61ce6ff491b9d497db42d900f50/users/6c828726ae814c0f879a5aa69f647a96/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:39.285432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.303037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 635/1249] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.306839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.350058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ea0ec-ea1c-4671-8a43-b21c8abdd032 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.368101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d52e7-34f2-40bc-b099-025ecc2f1d99 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.372837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.374153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.374880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.375163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.375905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:39.376212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.378226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.381235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:39.390457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 636/1250] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:39.391475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.398377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:39.401595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b997d-2657-4e8d-bc5f-470bb17141a7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.406379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.406938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.407217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.407488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.407969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.408101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.408608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.408887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.409157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.409796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:39.410088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.410214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.411674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a1e50-ceb1-479b-8b64-b786994cd389 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.423676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 615/1251] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:39.424956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.425161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.425477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.453575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] MFA Rules not processed for user `fd3620e3535b4040896736ce79e38fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:39.454908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:39.489953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 637/1252] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.490916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.519008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 616/1253] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:39.520001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.531880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e86058-8217-4e0a-a1aa-5a2aacee44c0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.565513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f4af-73d8-4c01-aba1-972617e89c91 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.570233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.570875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.571158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.571434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.572064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.574143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.591977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.593413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.593648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.593944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.594609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:39.594942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.596524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dcc4b6-b7d5-4dc1-a9b6-7f1ab6d0efad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.608299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 617/1254] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:39.609684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.610092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.610401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.629036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:39.630234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:39.637719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 618/1255] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:39.638462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.648191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb895a6-72db-422f-a2a5-04e4f235e91a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.652737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.653326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.653607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.653884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.654564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.656594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.679547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 638/1256] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.679547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.689230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86480742-9ae3-42ca-ae62-8cfb45ab1f04 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.693905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.694475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.694796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.695074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.695689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.697563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.761457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 619/1257] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.761457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.771572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b6f64-7970-40bf-91d2-0b2a5ccee152 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.776459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.777054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.778743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.778743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.778743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.780125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f5db3-22ea-4199-85c8-1c5bc0df64d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.787267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 639/1258] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.787267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.847564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7604a-071d-4713-acec-8263bfa0c07a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.853803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.854374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.854683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.854961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.855593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.857511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.882971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 620/1259] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.882971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c44ec-8997-4ebc-871b-1107181878d4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.882971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c44ec-8997-4ebc-871b-1107181878d4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.882971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c44ec-8997-4ebc-871b-1107181878d4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.883557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 621/1260] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:34:39.883955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c44ec-8997-4ebc-871b-1107181878d4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.911659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:39.947487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 640/1261] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/users => generated 282 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:39.949189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:39.950804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c44ec-8997-4ebc-871b-1107181878d4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:39.951177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:39.955928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:39.956564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:39.956890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:39.957161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:39.961798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:39.963989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:39.986798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:40.010371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13551c71-99cd-4091-96ed-c379c37c3401 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.049907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 622/1262] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:39 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:40.050850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.066253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.067077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.067358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.067703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.076847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.080565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.087744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 641/1263] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:40.088608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.097571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9c40f-2710-4af0-876e-efe435a51d97 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.116868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90e12c-d7a7-4f47-9fd3-a1fdf0b4b6be admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.121673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.122287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.122600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.122884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.124089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.125720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.136856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 623/1264] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.137658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.147054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3d098-b502-4b4d-89aa-52c933a38fb7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.151751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.152340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.161042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.161042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.161042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.161042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f33a0-3318-4309-9b0c-edd39f3b56d3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.161389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.161389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.161389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.161389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.161826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.161826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.174031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 642/1265] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.174031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.181084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea779d-4e17-440a-9622-133129e476bb admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.186029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.187735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.188024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.188527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.189174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.191111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.256490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 624/1266] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:40.256490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41627246-6417-4627-961e-59e340294131 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.256490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41627246-6417-4627-961e-59e340294131 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.256490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41627246-6417-4627-961e-59e340294131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.277199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 643/1267] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:40.277199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.285050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9972c-d92f-460d-8e48-9c0504d8c6ec admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.289897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.290554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.290842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.294690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.294690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.294690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.298531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-41627246-6417-4627-961e-59e340294131 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:40.299808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-41627246-6417-4627-961e-59e340294131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:40.305177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 644/1268] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.305963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.319909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3380d-7a3b-454f-9672-839b8c85b9af admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.325293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.326486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.326904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.327166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.327799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.329931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.376077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 625/1269] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:40.379166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41627246-6417-4627-961e-59e340294131 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.393623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41627246-6417-4627-961e-59e340294131 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.394817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 645/1270] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:40.395831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.400319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.400952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.401280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.401591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.402293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.404235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.406926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:40.410295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6875-3821-41be-b3b9-8f3079bad343 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.416815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.417370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 626/1271] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.417549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.417889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.418289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.419493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.419638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.421979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.432829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 646/1272] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.438303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.438549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2cb9-89a0-40bf-b8e5-746dc2dce569 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.443900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.444095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a69d4f-656d-4214-b93d-fb3bd6a0f73c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.444855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.445136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.445411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.446091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.447822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.452038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.452655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.452989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.453267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.453948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.456558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.459137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 627/1273] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.459975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.470543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa0bf-693a-4dc1-b769-ee7516f04a9c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.472927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 647/1274] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.475197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.475678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.475818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.476295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.476571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.477242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.478858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.489129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb63a-b3fa-4206-be93-8dae1c4e5cf6 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.490764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 628/1275] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.491591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.497942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.498610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.498899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.499185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.501163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc25d60-20a5-40a6-8a5f-f96aec45f2f1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.505702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.507177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.507177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.507177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.507604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.509171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.515422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:40.519583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 629/1276] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.520328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.530619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ef2b-ca50-40c2-aca0-1457ee86d7dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.537267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.538189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.538717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.539144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] PATH_INFO: `/v3/projects/a3e2936fe3344e60b6f3dc918b9c738e/users/020ff888a8a44d2eb6961829407637a9/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.558337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e2936fe3344e60b6f3dc918b9c738e, user_id=020ff888a8a44d2eb6961829407637a9, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.563551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.573105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.574783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.585005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 648/1277] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:40.585840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.597087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a51f2-b5e7-4601-8640-f01c9299bfaa service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.602025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.602726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.603066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.603356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] PATH_INFO: `/v3/projects/1a49e667714043b9a821ac5c0b8b3663/users/8665d60741af4d96a3ceb8f59ff458bb/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.619344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a49e667714043b9a821ac5c0b8b3663, user_id=8665d60741af4d96a3ceb8f59ff458bb, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.622118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.660122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 630/1278] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:40 2022] PUT /identity/v3/projects/a3e2936fe3344e60b6f3dc918b9c738e/users/020ff888a8a44d2eb6961829407637a9/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:40.660122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.724016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 649/1279] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:40 2022] PUT /identity/v3/projects/1a49e667714043b9a821ac5c0b8b3663/users/8665d60741af4d96a3ceb8f59ff458bb/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:40.725001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.734042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258734cb-52e1-4050-a0fe-c2c98ce0d550 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.797535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8da9-7287-49da-a367-1b80997589cf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.803947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.806140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.806246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.806342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.806342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.806481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.806481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.806673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.806758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.806758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.807451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.809094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.827127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 631/1280] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.827127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.827691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 650/1281] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.827691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.839540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b2d68-559f-4f5d-812b-5fadceb97cb8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.839740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11898810-675c-4163-8db9-1adfe997bf80 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.844876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.845621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.845936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.846261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.846858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.846986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.847887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.848372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.848529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747fa84-50b3-476d-9b4d-70ab8ae87a2c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.849035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] PATH_INFO: `/v3/projects/6ede276d10e247528cfda3008983bf6c/users/e0d6342874be479daf0b611b71b50ab8/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.860383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 651/1282] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:40.861655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58926973-009b-4ef3-b517-3f54e530b5fe None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.861945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58926973-009b-4ef3-b517-3f54e530b5fe None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.862222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58926973-009b-4ef3-b517-3f54e530b5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.868007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ede276d10e247528cfda3008983bf6c, user_id=e0d6342874be479daf0b611b71b50ab8, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.875310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5e593-becc-48b9-84b2-1f2570a23bd8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.883245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-58926973-009b-4ef3-b517-3f54e530b5fe None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:40.884452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-58926973-009b-4ef3-b517-3f54e530b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:40.893322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 652/1283] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:40.895891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.896253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.896738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.916678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:40.917897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:40.928280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 653/1284] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:40.929154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.941152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab68d7-1ae7-4672-b043-5406ffa392c5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:40.945701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:40.946354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.946673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.946947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] PATH_INFO: `/v3/projects/b10b04082888494d9ce6803298fc56bf/users/c65b8b8717b74354a44b438d6360c5e5/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.959005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 632/1285] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:40 2022] PUT /identity/v3/projects/6ede276d10e247528cfda3008983bf6c/users/e0d6342874be479daf0b611b71b50ab8/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:40.960513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b41c-df24-40ff-9ddc-9a1f5687fe3a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:40.960803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b41c-df24-40ff-9ddc-9a1f5687fe3a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:40.963471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b41c-df24-40ff-9ddc-9a1f5687fe3a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:40.963471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 633/1286] 10.222.0.39 () {54 vars in 912 bytes} [Wed Oct 19 13:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:34:40.963471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b41c-df24-40ff-9ddc-9a1f5687fe3a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:40.976025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10b04082888494d9ce6803298fc56bf, user_id=c65b8b8717b74354a44b438d6360c5e5, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:40.978804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:40.985948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:41.032238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b41c-df24-40ff-9ddc-9a1f5687fe3a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.036899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.037559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.037840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.038117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] PATH_INFO: `/v3/projects/738a7aeb1985442b84829882749eefcc/users/333cc4e4f1334ecb923f3583c18cfc11/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.055196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738a7aeb1985442b84829882749eefcc, user_id=333cc4e4f1334ecb923f3583c18cfc11, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.055825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 654/1287] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:40 2022] PUT /identity/v3/projects/b10b04082888494d9ce6803298fc56bf/users/c65b8b8717b74354a44b438d6360c5e5/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:41.056831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.059162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:41.111150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:41.130209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2eb2-84aa-42e6-a438-158f1b6a3993 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.134930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.135626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.135905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.136184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] PATH_INFO: `/v3/projects/8f5b23f416f54e5ca4763ba2b6562ed8/users/c4f083c1909243e89b719378605d8737/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.147300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 634/1288] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:40 2022] PUT /identity/v3/projects/738a7aeb1985442b84829882749eefcc/users/333cc4e4f1334ecb923f3583c18cfc11/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:41.148352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.161308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5b23f416f54e5ca4763ba2b6562ed8, user_id=c4f083c1909243e89b719378605d8737, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.164030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:41.204824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9008d8-815b-439f-9d39-f9213b465443 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.208486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.209091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.209399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.209673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.271371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.272810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cec28-c77e-43ea-852f-01cfc7af2e05 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:41.273715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 655/1289] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:41 2022] PUT /identity/v3/projects/8f5b23f416f54e5ca4763ba2b6562ed8/users/c4f083c1909243e89b719378605d8737/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:41.274620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.289461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 635/1290] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:41.290912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bb144-6ab0-4190-9580-43837f05a7c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.291260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bb144-6ab0-4190-9580-43837f05a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.291587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bb144-6ab0-4190-9580-43837f05a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.323503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cf5bb144-6ab0-4190-9580-43837f05a7c4 None None] MFA Rules not processed for user `e0d6342874be479daf0b611b71b50ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:41.324737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf5bb144-6ab0-4190-9580-43837f05a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:41.350190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccbdca-906a-46b4-a292-ad7696010f3f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.356063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.356792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.357172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.357538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.370615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:41.402560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 636/1291] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:41.403888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d63cf-e151-4447-b9fa-c39b2e9cdc9c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.404181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d63cf-e151-4447-b9fa-c39b2e9cdc9c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.404505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d63cf-e151-4447-b9fa-c39b2e9cdc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.434603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8f2d63cf-e151-4447-b9fa-c39b2e9cdc9c None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:41.435935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8f2d63cf-e151-4447-b9fa-c39b2e9cdc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:41.473251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.475241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11454-7968-444e-903f-ba143b33d42f service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:41.494627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 656/1292] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:41.496188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be520-ee00-4b65-88cb-0646997248fe None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.496652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be520-ee00-4b65-88cb-0646997248fe None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.498159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be520-ee00-4b65-88cb-0646997248fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.524353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 637/1293] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:41.534088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.534413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.535205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.548519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-429be520-ee00-4b65-88cb-0646997248fe None None] MFA Rules not processed for user `c65b8b8717b74354a44b438d6360c5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:41.549750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-429be520-ee00-4b65-88cb-0646997248fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:41.578843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] MFA Rules not processed for user `333cc4e4f1334ecb923f3583c18cfc11`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:41.581241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:41.646108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 657/1294] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:41.648402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.648843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.649238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.659812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:41.700963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 638/1295] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:41.703833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.705380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] MFA Rules not processed for user `c4f083c1909243e89b719378605d8737`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:41.709219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:41.732108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33146cd2-95ba-48fa-880d-746f8181e11b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.741703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.746773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.748482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.748482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.748794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.751114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:41.807812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 658/1296] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:41.808775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.859869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 639/1297] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:41.859869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:41.876937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377c88b-eee8-4d28-a8d2-c8e963408d6e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.914595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:41.956500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf150-f812-41ee-a5cb-fe2eaa5db462 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:41.961061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.961717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.962002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.962269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.969723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:41.971453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:41.971939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:41.972684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:41.974000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:41.978186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.059923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.061916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db362-3186-4060-8734-3dd15625e2e0 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.079822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 659/1298] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:42.083120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9bb-b09c-46f3-90fb-be8ea29295e7 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.083695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9bb-b09c-46f3-90fb-be8ea29295e7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.084186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9bb-b09c-46f3-90fb-be8ea29295e7 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.085720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 660/1299] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:34:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:34:42.086579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d9bb-b09c-46f3-90fb-be8ea29295e7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.105721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d9bb-b09c-46f3-90fb-be8ea29295e7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.113679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.114433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.114989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.115327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.138675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 640/1300] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:41 2022] POST /identity/v3/projects => generated 403 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:42.138675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.222310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd222dc-9d2f-46a6-a00b-bea3c02dde33 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.227904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.228770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.229289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.229655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.230411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.230593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.232618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b66f4f-c25c-41f4-88b5-a8f0d3f7e53c service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.247787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.259129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 661/1301] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:42.260689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f582c-992c-4661-987b-a958275eada6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.261409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f582c-992c-4661-987b-a958275eada6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.261649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f582c-992c-4661-987b-a958275eada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.301272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-a74f582c-992c-4661-987b-a958275eada6 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:42.306146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a74f582c-992c-4661-987b-a958275eada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:42.328630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:42.387392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 641/1302] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:42 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:42.391589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.427092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ce72-b34d-4666-a682-45bc2923010e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.446825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.448915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.448915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.448915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.448915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.450581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.496842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 662/1303] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:42.499143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f582c-992c-4661-987b-a958275eada6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.529407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f582c-992c-4661-987b-a958275eada6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.539379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.540017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.540302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.541118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.541355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.547178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.570378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 642/1304] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:42 2022] POST /identity/v3/users => generated 308 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:42.570378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.695105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 663/1305] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:42.695105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.695915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3e9ed-c9d9-4521-9ae6-2a8e7d2fd939 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.695915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.695915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.695915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.696513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.696700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2a8a-5db1-4aa4-bdc2-7a39946080e7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.696700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.696700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.696700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.697029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.697029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.702282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.717270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 664/1306] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:42.718491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.733748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff6632-8450-40ce-b6b2-a3d7488f7bfc admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.744914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.746024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.746590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.747628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] PATH_INFO: `/v3/projects/668a5402ab024c638b2512af9572d6bf/users/486ef9292abe4ecaa41190711951a846/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.784971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.786252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668a5402ab024c638b2512af9572d6bf, user_id=486ef9292abe4ecaa41190711951a846, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.791198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.793977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.809836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 643/1307] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:42.811599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.838735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c636e-5942-496d-bd11-3900d42ae1f5 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:42.850596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:42.851680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:42.852237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:42.852727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] PATH_INFO: `/v3/projects/5f5aba67273249bc9da90b4cb726c0f6/users/c4dceb1c35d64340aa9225e756bf3f06/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:42.872876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5aba67273249bc9da90b4cb726c0f6, user_id=c4dceb1c35d64340aa9225e756bf3f06, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:42.876755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:42.914065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 665/1308] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:42 2022] PUT /identity/v3/projects/668a5402ab024c638b2512af9572d6bf/users/486ef9292abe4ecaa41190711951a846/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:42.918275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:42.961688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:42.992462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 644/1309] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:42 2022] PUT /identity/v3/projects/5f5aba67273249bc9da90b4cb726c0f6/users/c4dceb1c35d64340aa9225e756bf3f06/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:42.993524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:43.025052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:43.031582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34361c-0308-47eb-8965-9cf36085c906 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.113119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc9d10-f61c-42bf-8acc-74ac9074b1ec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.126280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.126942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.127065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.127171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.127263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.127455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.127590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.127735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.128339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.128552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.130140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.130270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.142992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 666/1310] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:43.147044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:43.159810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6bf59-39da-4640-8ad0-d5a87aa80331 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.161704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 645/1311] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:43.164958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:43.165952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.166831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.167187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.167550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] PATH_INFO: `/v3/projects/668a5402ab024c638b2512af9572d6bf/users/486ef9292abe4ecaa41190711951a846/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.170755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=668a5402ab024c638b2512af9572d6bf, user_id=486ef9292abe4ecaa41190711951a846, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.186866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12765389-14bd-4257-9647-ed76c1530c36 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.188190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37930851-3690-4484-aa84-10f0fdee9dad admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.192937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.194646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.195196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.195438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] PATH_INFO: `/v3/projects/5f5aba67273249bc9da90b4cb726c0f6/users/c4dceb1c35d64340aa9225e756bf3f06/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.199044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5aba67273249bc9da90b4cb726c0f6, user_id=c4dceb1c35d64340aa9225e756bf3f06, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.202948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442260bd-e68d-496d-a65f-34367455384f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.204500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 667/1312] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:43 2022] PUT /identity/v3/projects/668a5402ab024c638b2512af9572d6bf/users/486ef9292abe4ecaa41190711951a846/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:43.206228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e436-e90d-4028-bd7f-c6ddd4716094 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.206886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e436-e90d-4028-bd7f-c6ddd4716094 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.207353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3e436-e90d-4028-bd7f-c6ddd4716094 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.208899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 668/1313] 10.222.0.39 () {54 vars in 912 bytes} [Wed Oct 19 13:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:34:43.215607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe95e8f-331d-451e-88c9-693e6070355b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.216207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe95e8f-331d-451e-88c9-693e6070355b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.216953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe95e8f-331d-451e-88c9-693e6070355b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.224075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 646/1314] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:43 2022] PUT /identity/v3/projects/5f5aba67273249bc9da90b4cb726c0f6/users/c4dceb1c35d64340aa9225e756bf3f06/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:43.226234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.226933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.227399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.276472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ebe95e8f-331d-451e-88c9-693e6070355b None None] MFA Rules not processed for user `486ef9292abe4ecaa41190711951a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:43.279174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ebe95e8f-331d-451e-88c9-693e6070355b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:43.444014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 669/1315] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:43.444014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.444014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.444014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.491770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] MFA Rules not processed for user `c4dceb1c35d64340aa9225e756bf3f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:43.500255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:43.519439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:43.520583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:43.601755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:43.662324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 647/1316] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:43.663273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:43.726859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 670/1317] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:43.787289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c04cf-5266-4ea6-8c69-f7e77a6ab9f8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.798997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.799988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.800448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.800853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.823304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.829660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.856702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 648/1318] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:43.871299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:43.908017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4b065-f57b-4724-9858-8f5020441c5e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:43.913289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:43.914278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:43.914685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:43.921291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:43.950866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:43.960375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:43.994236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:44.011846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f31002-4fbc-4bd4-ab11-cc864c8bc370 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:44.013664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 671/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:44.039905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:44.039905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:44.039905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:44.039905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:44.064474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:44.067859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:44.089221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 649/1320] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:43 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:44.397799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:44.414108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67973274-f11a-4909-81d1-099f63ee18b2 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:44.421031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:44.422356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:44.422694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:44.433152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:44.519915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:44.523166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa037ec-29d4-4ed2-a240-fb3b893de5ef service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:44.540398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 672/1321] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:45.289610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:45.323591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:45.415064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957c2ac-e502-4a8f-8238-27542024e91a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:45.422004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:45.422702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:45.422980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:45.423258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] PATH_INFO: `/v3/users/333cc4e4f1334ecb923f3583c18cfc11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:45.424473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333cc4e4f1334ecb923f3583c18cfc11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:45.425503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:34:45.425503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Oct 19 13:34:45.425503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: Oct 19 13:34:45.426958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:45.648873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333cc4e4f1334ecb923f3583c18cfc11'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:45.649816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333cc4e4f1334ecb923f3583c18cfc11'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:45.669683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333cc4e4f1334ecb923f3583c18cfc11'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:45.710219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333cc4e4f1334ecb923f3583c18cfc11'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:45.829850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333cc4e4f1334ecb923f3583c18cfc11'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:45.868942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 650/1322] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:34:45 2022] DELETE /identity/v3/users/333cc4e4f1334ecb923f3583c18cfc11 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:45.874399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:45.875844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:45.877401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:45.907718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:45.909923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:46.029317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 673/1323] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:46.042499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:46.044095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:46.166016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82affd7-0861-44c2-8c33-dbdfb46452d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:46.171377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:46.172079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:46.172362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:46.173707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:46.184253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:46.185546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:46.205382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 651/1324] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:48.165793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:48.167643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:48.227071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb5071-2579-4148-ae32-965b82fd7c4e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:48.255538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:48.256233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:48.259216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:48.259987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] PATH_INFO: `/v3/projects/738a7aeb1985442b84829882749eefcc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:48.265856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738a7aeb1985442b84829882749eefcc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:48.270689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: WARNING py.warnings [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 13:34:48.270689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: warnings.warn(msg) Oct 19 13:34:48.270689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: Oct 19 13:34:48.274222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:48.555290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738a7aeb1985442b84829882749eefcc', 'request_id': 'req-6f87a3b7-2ac4-4711-b0a6-21113228fba4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:48.560883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:48.564272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738a7aeb1985442b84829882749eefcc', 'request_id': 'req-6f87a3b7-2ac4-4711-b0a6-21113228fba4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:48.706945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738a7aeb1985442b84829882749eefcc', 'request_id': 'req-6f87a3b7-2ac4-4711-b0a6-21113228fba4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:48.742978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] The token cache is being invalidate because project 738a7aeb1985442b84829882749eefcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:34:48.744503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:48.750809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6f87a3b7-2ac4-4711-b0a6-21113228fba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:48.757408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:48.758595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 674/1325] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:34:48 2022] DELETE /identity/v3/projects/738a7aeb1985442b84829882749eefcc => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:48.773565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b5c1-cf1a-4916-944b-c0a10b753356 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:48.867875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:48.924388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:48.925144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:48.926112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:48.926489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] PATH_INFO: `/v3/users/c4dceb1c35d64340aa9225e756bf3f06` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:48.930179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4dceb1c35d64340aa9225e756bf3f06)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:48.932247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:49.152725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4dceb1c35d64340aa9225e756bf3f06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:49.153864 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4dceb1c35d64340aa9225e756bf3f06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:49.175524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4dceb1c35d64340aa9225e756bf3f06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:49.198001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4dceb1c35d64340aa9225e756bf3f06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:49.255899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4dceb1c35d64340aa9225e756bf3f06'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:49.263610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 652/1326] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:34:48 2022] DELETE /identity/v3/users/c4dceb1c35d64340aa9225e756bf3f06 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:49.409094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:49.410061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:49.410656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:49.473301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] MFA Rules not processed for user `6c828726ae814c0f879a5aa69f647a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:49.473301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:49.575106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 675/1327] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:49.595916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:49.597424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:49.756634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6efa-f457-40b7-b8b0-b1a124874006 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:49.764525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:49.765979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:49.780447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:49.780956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:49.802189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:49.804119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369d86b-ae3e-4a5d-8733-a0bf0a392b64 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:49.835247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 653/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:50.960226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:50.961792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:51.227211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaab8a-b4c2-41ce-829b-f3d333c0b226 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:51.243873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:51.244947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:51.245383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:51.245835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] PATH_INFO: `/v3/projects/5f5aba67273249bc9da90b4cb726c0f6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:51.247052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5aba67273249bc9da90b4cb726c0f6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:51.249053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:51.452271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5aba67273249bc9da90b4cb726c0f6', 'request_id': 'req-5b79e1de-f502-4ed2-9214-6b62e0b68275'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:51.461837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5aba67273249bc9da90b4cb726c0f6', 'request_id': 'req-5b79e1de-f502-4ed2-9214-6b62e0b68275'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:51.528322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5aba67273249bc9da90b4cb726c0f6', 'request_id': 'req-5b79e1de-f502-4ed2-9214-6b62e0b68275'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:51.551962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] The token cache is being invalidate because project 5f5aba67273249bc9da90b4cb726c0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:34:51.552560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:51.554765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5b79e1de-f502-4ed2-9214-6b62e0b68275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:51.560075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 676/1329] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:34:50 2022] DELETE /identity/v3/projects/5f5aba67273249bc9da90b4cb726c0f6 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:51.564297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:51.564810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:51.565057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:51.593188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:51.595753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:51.665056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:51.703272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:51.703855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:51.704295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:51.705445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 654/1330] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:51.719268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:51.776580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:51.782063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:51.811315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 677/1331] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:51.811465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:51.827460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dc16f-1992-4dcb-bb5b-d75e7d3cd4ae None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:51.851685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:51.860528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:51.860856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:51.866615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:51.867382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:51.867697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:51.873459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:51.927513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 655/1332] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:51.927998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:51.941298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcebcd2-a1c6-4bf9-8b82-7689941b36ed None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:51.961150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:51.962672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:51.967689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:51.967988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:51.973456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:51.977347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:52.160471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 678/1333] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:51 2022] POST /identity/v3/projects => generated 355 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:34:52.163805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:52.176869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6375-0f5b-4a77-8992-b9ba7d986ab8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:52.383355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:52.385689 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:52.387582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:52.388015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:52.388874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:52.393331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:52.552790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 656/1334] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:52 2022] POST /identity/v3/users => generated 278 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:52.553895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:52.587440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb80f4-2b6f-4dbf-91d7-06edc9ca544a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:52.597645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:52.598432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:52.598846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:52.599222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:52.600000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:52.601742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:52.627085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 679/1335] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:52.631160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:52.647857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d83d1-75c0-4101-9633-2a65d226108a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:52.661227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:52.661227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:52.661227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:52.661227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:52.661658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:52.661658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b55072-6212-46e2-9d6a-b9bb0451856e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:52.684015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 657/1336] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:52.688578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:52.709243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68692450-a886-42a3-8d39-b935a648d4e2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:52.722439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:52.723372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:52.723664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:52.723948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] PATH_INFO: `/v3/projects/c3aadaa99dd54ed59d4d752d0804e39d/users/dc358c517c7346caba17ec870678ff16/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:52.789190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3aadaa99dd54ed59d4d752d0804e39d, user_id=dc358c517c7346caba17ec870678ff16, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:52.789190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:52.913130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 680/1337] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:52 2022] PUT /identity/v3/projects/c3aadaa99dd54ed59d4d752d0804e39d/users/dc358c517c7346caba17ec870678ff16/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:52.926072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7daae-5ba5-41ac-99d9-c50722630978 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:52.926975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7daae-5ba5-41ac-99d9-c50722630978 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:52.927171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7daae-5ba5-41ac-99d9-c50722630978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.003810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-daf7daae-5ba5-41ac-99d9-c50722630978 None None] MFA Rules not processed for user `dc358c517c7346caba17ec870678ff16`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:53.005291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-daf7daae-5ba5-41ac-99d9-c50722630978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:53.183668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 658/1338] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:53.189988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:53.267228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5669-9612-479f-a681-4d39d5284a67 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:53.267929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5669-9612-479f-a681-4d39d5284a67 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:53.268392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5669-9612-479f-a681-4d39d5284a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.314021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5745405-26df-42d4-9065-a23da3546cef admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:53.327373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:53.328516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:53.329047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:53.329560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.330568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:53.340074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:53.346166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b71f5669-9612-479f-a681-4d39d5284a67 None None] MFA Rules not processed for user `020ff888a8a44d2eb6961829407637a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:53.351674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b71f5669-9612-479f-a681-4d39d5284a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:53.451045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 681/1339] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:53 2022] POST /identity/v3/projects => generated 357 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:53.452358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:53.480613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:53.563709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 659/1340] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:53.565182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fd07-94d6-480b-9af3-c3de4d508d4b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:53.565474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fd07-94d6-480b-9af3-c3de4d508d4b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:53.565867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132fd07-94d6-480b-9af3-c3de4d508d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.621074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a132fd07-94d6-480b-9af3-c3de4d508d4b None None] MFA Rules not processed for user `020ff888a8a44d2eb6961829407637a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:53.623551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a132fd07-94d6-480b-9af3-c3de4d508d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:53.679767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33070605-2d67-43d0-8d46-ccab29997de5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:53.685426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:53.686355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:53.686838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:53.687525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.688339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:53.691677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:53.731821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 660/1341] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:53.750173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:53.750468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:53.757107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:53.821914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 682/1342] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:53 2022] POST /identity/v3/users => generated 285 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:53.825034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:53.827430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:53.830203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] MFA Rules not processed for user `8665d60741af4d96a3ceb8f59ff458bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:53.833257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:53.866715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:54.001578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207e3fd-1a4f-4392-9f48-18ddb349fd18 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.008541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.009329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.009796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.010148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.025944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 661/1343] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:54.026673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.027316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.028583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2b253-272a-444d-92a2-1fec4c9923ee service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.042201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 683/1344] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:54.044151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.044531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.044958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.072217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] MFA Rules not processed for user `8665d60741af4d96a3ceb8f59ff458bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:54.073537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:54.095479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 684/1345] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:54.121264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc80fe-e5fa-4ae7-80a1-9ef76a68f5df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.141151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.144385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.144675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.146152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.148585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.153814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.182714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 662/1346] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:54.184968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.231360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907a0b7-06f8-412c-b7cd-165f6ff96dab None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.232251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.232251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:54.260013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.260716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.261000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.261278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] PATH_INFO: `/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3/users/444afc6c39fe4d35acfe2b227f64e675/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.275978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db30ea-5ad1-483e-946b-ab1c102f7840 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.303184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.303835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.304121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.304415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.309601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61eb9c1e6c6e4174bc9b176a31320bf3, user_id=444afc6c39fe4d35acfe2b227f64e675, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.316170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.318392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.319144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.333632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 663/1347] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:54.438943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 685/1348] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:54 2022] PUT /identity/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3/users/444afc6c39fe4d35acfe2b227f64e675/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:54.443170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.470041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d0185-0451-4fd5-a0c0-db9033f3a423 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.547529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:54.587211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.588836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.589482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.589948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.590978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.594657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7f5c9-c89d-47e9-b86f-0881b9ba3b6c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.621406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.624417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 664/1349] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:54.640008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e3298-9d6d-4303-8166-f29f19dc638b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.649624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.651390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.652820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.655230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] PATH_INFO: `/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3/users/444afc6c39fe4d35acfe2b227f64e675/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.664729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61eb9c1e6c6e4174bc9b176a31320bf3, user_id=444afc6c39fe4d35acfe2b227f64e675, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:54.671555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5aeb67-3f07-49cd-a403-bf74aa4a2a1e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:54.707581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 686/1350] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:54 2022] PUT /identity/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3/users/444afc6c39fe4d35acfe2b227f64e675/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:34:54.707813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.708313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.709468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.750792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] MFA Rules not processed for user `444afc6c39fe4d35acfe2b227f64e675`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:54.753121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:54.831818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c349b8-e194-4e79-98b8-72f984b9e7c9 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.832408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c349b8-e194-4e79-98b8-72f984b9e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.832946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c349b8-e194-4e79-98b8-72f984b9e7c9 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.834601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 687/1351] 10.222.0.39 () {54 vars in 912 bytes} [Wed Oct 19 13:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:34:54.844336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab912-c528-4398-a3ef-fa0601bb9555 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.844804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab912-c528-4398-a3ef-fa0601bb9555 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.845227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab912-c528-4398-a3ef-fa0601bb9555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:54.850399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 665/1352] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:54.862764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:54.865681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:54.905252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8b8ab912-c528-4398-a3ef-fa0601bb9555 None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:54.906471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8b8ab912-c528-4398-a3ef-fa0601bb9555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:54.989497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253e4f-d4d8-4ce9-ab51-69d565551f63 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:54.993562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:54.994184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:54.994467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:54.994774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.008106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 688/1353] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:55.010348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:55.010797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.011295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.011917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629eaaa-24d1-480d-a63a-a762299a3924 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:55.012201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.014803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 689/1354] 10.222.0.39 () {54 vars in 912 bytes} [Wed Oct 19 13:34:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:34:55.015828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:55.017791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.026335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 666/1355] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:34:55.027722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476e5f-7c58-4412-813c-18d88beda699 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.028259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476e5f-7c58-4412-813c-18d88beda699 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.028606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476e5f-7c58-4412-813c-18d88beda699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.034824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7a849-5770-404d-ba25-568e64001b05 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:55.049443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:55.050399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.050926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.051337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.066772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-35476e5f-7c58-4412-813c-18d88beda699 None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:55.067321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:55.069164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35476e5f-7c58-4412-813c-18d88beda699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.069425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:55.077570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 667/1356] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:55.080161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.080520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.080962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.083852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 690/1357] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:55.093082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:55.097561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.108827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] MFA Rules not processed for user `6c828726ae814c0f879a5aa69f647a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:55.110152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.116040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf39c07-be6e-411a-b5e7-01773fd2cf86 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:55.125973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:55.126651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.126955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.127210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.145915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:55.147650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:55.166295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 691/1358] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:55.194543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:55.204309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 668/1359] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:55.348803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:55.354772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.371228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8db3-e583-4b08-8df5-b7e2ecea96a1 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:55.392588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:55.393257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.393637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.393918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.435116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:55.497204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:55.500146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b4d8d-6c4d-4e6a-8a7c-ae9e24604079 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:55.516525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 692/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:55.726391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:55.727873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.894413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf628c27-8a58-4fd8-887f-31bd7f409533 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:55.906348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:55.907024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.907301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.907572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.911171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.919388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b322b-fe2e-4314-a379-d014d4d203f0 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.920832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b322b-fe2e-4314-a379-d014d4d203f0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.921344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b322b-fe2e-4314-a379-d014d4d203f0 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.923452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 693/1361] 10.222.0.39 () {54 vars in 961 bytes} [Wed Oct 19 13:34:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:34:55.929772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:55.930058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:55.930336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:55.938103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:55.940494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:55.953584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 669/1362] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:55.970403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:55.973735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:55.988304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 694/1363] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Oct 19 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:55.995974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:56.011358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef657e9-e877-47bb-8e7c-e0864ba12fc1 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:56.019764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:56.020722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:56.021364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:56.021825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:56.034849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:56.037500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca95452-85d5-46c8-b38a-73bfc6b70e07 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:56.069270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 670/1364] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:56.260605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:56.262109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:56.519519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817f870-a0ff-4053-b46e-f96b14ef3543 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:56.544730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:56.545954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:56.546459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:56.547013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] PATH_INFO: `/v3/users/c65b8b8717b74354a44b438d6360c5e5` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:56.550564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65b8b8717b74354a44b438d6360c5e5)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:56.553420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:56.563761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e943a9-e076-4bde-87af-177bd3476c40 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:56.564053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e943a9-e076-4bde-87af-177bd3476c40 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:56.564333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e943a9-e076-4bde-87af-177bd3476c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:56.613216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-03e943a9-e076-4bde-87af-177bd3476c40 None None] MFA Rules not processed for user `020ff888a8a44d2eb6961829407637a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:56.616641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03e943a9-e076-4bde-87af-177bd3476c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:56.741618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 671/1365] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:57.071810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65b8b8717b74354a44b438d6360c5e5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:57.071810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65b8b8717b74354a44b438d6360c5e5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:57.071810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65b8b8717b74354a44b438d6360c5e5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:57.071810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65b8b8717b74354a44b438d6360c5e5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:57.107221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65b8b8717b74354a44b438d6360c5e5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:57.119016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 695/1366] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:34:56 2022] DELETE /identity/v3/users/c65b8b8717b74354a44b438d6360c5e5 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:57.125135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:57.125135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:57.139714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:57.215252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:57.220067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:57.452292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 672/1367] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:57.463146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:57.465042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:57.594262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:57.655189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8c1a-58e3-4b91-b1d9-e0fadd1bbd4d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:57.669137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:57.669137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:57.669137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:57.669137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:57.691505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:57.693197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6547ba8-ecda-479a-bf8f-8d90c2919a13 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:57.742709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 696/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:34:58.725816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:58.730458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:58.753685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e3ba-bf64-48bf-b207-9e0126235676 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:58.764455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:58.765130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:58.765411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:58.765692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] PATH_INFO: `/v3/projects/b10b04082888494d9ce6803298fc56bf` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:58.770751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10b04082888494d9ce6803298fc56bf)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:58.772697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:58.824670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:34:59.015718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10b04082888494d9ce6803298fc56bf', 'request_id': 'req-e121df63-8f77-41d1-9579-a51b6cfb3084'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:59.040746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10b04082888494d9ce6803298fc56bf', 'request_id': 'req-e121df63-8f77-41d1-9579-a51b6cfb3084'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:59.111886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10b04082888494d9ce6803298fc56bf', 'request_id': 'req-e121df63-8f77-41d1-9579-a51b6cfb3084'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:59.133823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] The token cache is being invalidate because project b10b04082888494d9ce6803298fc56bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:34:59.134002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:59.134670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e121df63-8f77-41d1-9579-a51b6cfb3084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:34:59.136256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 673/1369] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:34:58 2022] DELETE /identity/v3/projects/b10b04082888494d9ce6803298fc56bf => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:34:59.142633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.143148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.143233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.165016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:59.166412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:59.242079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 697/1370] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:34:59.248502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.249777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.249777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.283592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:34:59.285232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:34:59.305081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 674/1371] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:34:59.305405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.316789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42000b-dccf-4321-9807-265d43ba4e35 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.322893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.324004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.329024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.329024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.329024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:34:59.329024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.329024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:59.349042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.359513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 698/1372] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:34:59.363699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a9a6b-3d92-472d-abcb-2473fa9ce495 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.375482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.378242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.378627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.380370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.381115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.398339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:59.535463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 675/1373] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:34:59 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:59.539639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.549874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5081a3-e957-4011-93e8-1d37aea8b339 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.622853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.623716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.623998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.624275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.624910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.627446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:59.741123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 699/1374] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:34:59 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:34:59.745381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.762630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744ba9-b71b-47dc-853d-af6e9c48c378 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.764325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.765020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.765506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.765966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.767257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.769082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:59.807023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 676/1375] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:59.807506 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.822431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48413-3fe9-4135-b02f-708a6532be64 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.828210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.829024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.829400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.829851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.831690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.833409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aed572-7a71-484e-9d05-cb95adcca9f9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:34:59.850402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 700/1376] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:34:59.851473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:34:59.866201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895efa2f-2caf-462a-80bf-ad313c4fda56 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:34:59.872432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:34:59.873949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:34:59.874403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:34:59.875205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] PATH_INFO: `/v3/projects/dd3b72f7df934a1bb063d6e63dc2d030/users/a508f4d372604579b582587dbd6c9865/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:34:59.894025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3b72f7df934a1bb063d6e63dc2d030, user_id=a508f4d372604579b582587dbd6c9865, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:34:59.897440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.010319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 677/1377] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:34:59 2022] PUT /identity/v3/projects/dd3b72f7df934a1bb063d6e63dc2d030/users/a508f4d372604579b582587dbd6c9865/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:00.018324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.018918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.019194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.047798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] MFA Rules not processed for user `a508f4d372604579b582587dbd6c9865`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:00.049404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:00.105390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:00.176079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 701/1378] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:00.180331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.257902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945242-7a8e-4d9b-b1aa-75b4be47045c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.262607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.263796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.264075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.264346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.264992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.267072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.353366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 678/1379] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:00 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:00.359716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.367210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c36fe-eb39-4c23-b67a-5b7ab7014b9f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.499416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.500069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.500414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.500932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.501967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.505512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.649626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 702/1380] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:00 2022] POST /identity/v3/users => generated 300 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:00.653861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.664621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f936-2783-41e0-aeef-097b89ba7368 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.669802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.670478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.670712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.670990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.671632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.673267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.685889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 679/1381] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:00.690371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.702423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975aa0-2eb4-46ee-8d87-98d4aa9b21bc admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.707396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.708086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.708366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.708641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] PATH_INFO: `/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2/users/fb8e6b56e2fd4050885ff6ecb4ed55cb/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.725614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81be88bbe33d4f68b8a1482f7fbf58c2, user_id=fb8e6b56e2fd4050885ff6ecb4ed55cb, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.728336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.814240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 703/1382] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:00 2022] PUT /identity/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2/users/fb8e6b56e2fd4050885ff6ecb4ed55cb/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:00.815623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.827087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971574f-bc21-41e0-bf9e-e4284c7354ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.844112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:00.907033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.907824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.908202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.908578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.909476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.911898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f09366-579e-4b33-ae90-31faca690d39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.925183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 680/1383] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:00.929273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:00.940450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab126c-2b63-41c3-80dd-b5bcb3ca56c3 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:00.945599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:00.946290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.946631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.946919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] PATH_INFO: `/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2/users/fb8e6b56e2fd4050885ff6ecb4ed55cb/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:00.948913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81be88bbe33d4f68b8a1482f7fbf58c2, user_id=fb8e6b56e2fd4050885ff6ecb4ed55cb, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:00.951841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:00.964673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 704/1384] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:00 2022] PUT /identity/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2/users/fb8e6b56e2fd4050885ff6ecb4ed55cb/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:00.975770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:00.976054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:00.976484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:01.021768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] MFA Rules not processed for user `fb8e6b56e2fd4050885ff6ecb4ed55cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:01.023108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:01.123724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 681/1385] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:01.136451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:01.139197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:01.225521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef4de-7b7a-4c4e-83cb-b63f155abf37 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:01.231923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:01.232722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:01.233109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:01.235155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:01.245252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:01.247260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3212e-5e12-4108-ad71-8129b28bea4d service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:01.258808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 705/1386] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:01.294652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:01.390769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c2e7b-8415-4e1e-b7fa-bdab8a9b444f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:01.407029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:01.409697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:01.411733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:01.411733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] PATH_INFO: `/v3/users/a508f4d372604579b582587dbd6c9865` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:01.413289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a508f4d372604579b582587dbd6c9865)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:01.423422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:01.658690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:01.666011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a508f4d372604579b582587dbd6c9865'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:01.666961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a508f4d372604579b582587dbd6c9865'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:01.681221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a508f4d372604579b582587dbd6c9865'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:01.693865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a508f4d372604579b582587dbd6c9865'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:01.763907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a508f4d372604579b582587dbd6c9865'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:01.770434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 682/1387] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:01 2022] DELETE /identity/v3/users/a508f4d372604579b582587dbd6c9865 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:01.774898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:01.775186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:01.775461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:01.799634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:01.801050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:01.831891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:01.867526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 706/1388] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:01.877762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:01.879399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:01.987877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84615c7d-4c48-46e6-9a3b-9bc603d2d3f8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:01.992429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:01.993263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:01.993547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:01.993826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:02.009649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:02.011719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:02.023685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 683/1389] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:02.817465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:02.819952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:02.838356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e32a3e-5907-46d8-b0fc-b1cfb2f9a9b9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:02.845505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:02.846367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:02.846853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:02.847324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] PATH_INFO: `/v3/projects/dd3b72f7df934a1bb063d6e63dc2d030` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:02.849016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3b72f7df934a1bb063d6e63dc2d030)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:02.851530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:02.988514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3b72f7df934a1bb063d6e63dc2d030', 'request_id': 'req-cdb19e39-5850-497a-9368-dad7fdf63083'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.005012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3b72f7df934a1bb063d6e63dc2d030', 'request_id': 'req-cdb19e39-5850-497a-9368-dad7fdf63083'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.087898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3b72f7df934a1bb063d6e63dc2d030', 'request_id': 'req-cdb19e39-5850-497a-9368-dad7fdf63083'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.112067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] The token cache is being invalidate because project dd3b72f7df934a1bb063d6e63dc2d030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:03.112574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.113691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cdb19e39-5850-497a-9368-dad7fdf63083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.118756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 707/1390] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:02 2022] DELETE /identity/v3/projects/dd3b72f7df934a1bb063d6e63dc2d030 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:03.119829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:03.146547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed514d66-63f0-45a4-87a6-af1922fd83f7 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:03.289786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:03.290555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:03.290848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:03.291300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] PATH_INFO: `/v3/users/fb8e6b56e2fd4050885ff6ecb4ed55cb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:03.295156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8e6b56e2fd4050885ff6ecb4ed55cb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:03.297643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:03.615658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8e6b56e2fd4050885ff6ecb4ed55cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.615658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8e6b56e2fd4050885ff6ecb4ed55cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.624779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8e6b56e2fd4050885ff6ecb4ed55cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.632904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:03.645004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8e6b56e2fd4050885ff6ecb4ed55cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.757641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8e6b56e2fd4050885ff6ecb4ed55cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:03.774959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 684/1391] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:03 2022] DELETE /identity/v3/users/fb8e6b56e2fd4050885ff6ecb4ed55cb => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:03.798457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:03.798795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:03.799081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:03.916117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:03.918559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:04.122134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 708/1392] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:35:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:04.523550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:04.529343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:04.604416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:04.606049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:04.656980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668258f-3f34-43d3-ba0f-b56dd2f2176c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:04.659191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:04.671288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:04.672972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:04.672972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:04.672972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] PATH_INFO: `/v3/users/fd3620e3535b4040896736ce79e38fd7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:04.673907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3620e3535b4040896736ce79e38fd7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:04.677040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:04.815721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89346162-9b35-4401-a7bd-b296be78fe3e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:04.824981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:04.826153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:04.826438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:04.826753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:04.837083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:04.852385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:04.876337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 709/1393] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:35:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 13:35:04.877256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:04.881710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:04.901342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78540b3e-1f39-4b99-a43b-104b08c8bbec service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:04.911255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:04.912699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:04.914491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:04.915199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] PATH_INFO: `/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:04.917572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81be88bbe33d4f68b8a1482f7fbf58c2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:04.920372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:05.052813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3620e3535b4040896736ce79e38fd7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.056965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3620e3535b4040896736ce79e38fd7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.097917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3620e3535b4040896736ce79e38fd7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.110754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81be88bbe33d4f68b8a1482f7fbf58c2', 'request_id': 'req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.132842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81be88bbe33d4f68b8a1482f7fbf58c2', 'request_id': 'req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.159875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3620e3535b4040896736ce79e38fd7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.233109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81be88bbe33d4f68b8a1482f7fbf58c2', 'request_id': 'req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.256541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] The token cache is being invalidate because project 81be88bbe33d4f68b8a1482f7fbf58c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:05.256848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.257572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b4b3318-3f6f-4055-9acb-aeaa1ccdc454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.297103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 710/1394] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:04 2022] DELETE /identity/v3/projects/81be88bbe33d4f68b8a1482f7fbf58c2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:05.303242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5098d-0b5a-4ca6-8f1f-7555ef73fe1f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.303557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5098d-0b5a-4ca6-8f1f-7555ef73fe1f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.303920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5098d-0b5a-4ca6-8f1f-7555ef73fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.338651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-53a5098d-0b5a-4ca6-8f1f-7555ef73fe1f None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:05.340734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-53a5098d-0b5a-4ca6-8f1f-7555ef73fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.385253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ab12d4ab-782c-43e8-b1bf-a56aabc2a711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3620e3535b4040896736ce79e38fd7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:05.394830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 685/1395] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:04 2022] DELETE /identity/v3/users/fd3620e3535b4040896736ce79e38fd7 => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:05.394830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.394830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.394830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.460424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:05.463120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.472733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 711/1396] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 19 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:05.474930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.475219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.475502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.507649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:05.509196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.541645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:05.593270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:05.609588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 712/1397] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:05.610636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:05.615071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.621155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 686/1398] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:05.622284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:05.627404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.779538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33456918-efb0-43ea-888d-bfe6dfb962ff None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:05.785263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:05.785844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.786130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.786407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.828254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:05.828254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf809de-20ea-4b39-991e-b8c872e2cf07 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:05.839548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 713/1399] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:05.841843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.842328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.842639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.875420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29674c01-1e9f-4b04-8bb9-7162e9569a7f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:05.879513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:05.879724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:05.880569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.880800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:05.880896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.881127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.894559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 714/1400] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:05.898445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:05.915610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:05.917300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:05.932325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd69224-5d94-4fb0-88eb-3099af486e39 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:05.953926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:05.953926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:05.953926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:05.958845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 687/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:05.970116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:05.974020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:05.974304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:05.984123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:06.041915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 715/1402] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:06.043219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:06.055468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41f64-cda7-449f-bc3f-fc156180d01c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:06.064837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:06.065699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:06.066230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:06.066661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:06.067955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:06.070044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:06.396896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 688/1403] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:06 2022] POST /identity/v3/projects => generated 409 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:06.401589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:06.437874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849d73-e56e-4afe-b029-d1493dba11bf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:06.618239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:06.618835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:06.619124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:06.619406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:06.620199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:06.623700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:06.748608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 716/1404] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:06 2022] POST /identity/v3/users => generated 305 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:06.753007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:06.775258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6c3c6-e58b-498d-ac75-42d16a77e64f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:06.805489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:06.806219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:06.806631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:06.806985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:06.807735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:06.809718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:06.850035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 689/1405] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:06.852996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:06.887431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3662-d153-43a6-a469-595a4715117c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:06.888847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:06.889640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:06.890066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:06.890493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:06.891304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:06.893075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fd98e-81ab-4889-96a5-0cfbd42242bb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:06.910271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 717/1406] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:06.911837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:06.927477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89df01-b37b-4135-91f4-9e47894635af admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:06.936150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:06.937462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:06.938089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:06.938567 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] PATH_INFO: `/v3/projects/434398e05fd246f5b6cd87e874a9b7ac/users/fb06f0df47934195aad5e10d2489ff12/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:06.959486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=434398e05fd246f5b6cd87e874a9b7ac, user_id=fb06f0df47934195aad5e10d2489ff12, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:06.962362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:07.044492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 690/1407] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:06 2022] PUT /identity/v3/projects/434398e05fd246f5b6cd87e874a9b7ac/users/fb06f0df47934195aad5e10d2489ff12/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:07.057125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:07.057125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:07.057673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:07.108073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] MFA Rules not processed for user `fb06f0df47934195aad5e10d2489ff12`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:07.111361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:07.141895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:07.233421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:07.234548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 718/1408] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:07.368280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad93e1-6ce5-429c-bc53-3e7dfa7076ff admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:07.375043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:07.376300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:07.376487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:07.376837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:07.377717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:07.379843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:07.443305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:07.446443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:07.497067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd85bb7-ebd0-4d6b-a0b2-5df0d816ed04 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:07.505437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 691/1409] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:07 2022] POST /identity/v3/projects => generated 407 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:07.508902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:07.520248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:07.521100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:07.541180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:07.541180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] PATH_INFO: `/v3/projects/ddae0cb1d13b4d4bbbcb2ba46a39f6d0` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:07.543993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddae0cb1d13b4d4bbbcb2ba46a39f6d0)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:07.548784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:07.588471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:07.637652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c59108-31ef-4af8-b4fe-527059537130 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:07.644850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:07.647152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:07.647852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:07.648624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:07.649751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:07.655067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:07.715281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddae0cb1d13b4d4bbbcb2ba46a39f6d0', 'request_id': 'req-4abdc575-6950-42eb-9ee8-31910479a44c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:07.731232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddae0cb1d13b4d4bbbcb2ba46a39f6d0', 'request_id': 'req-4abdc575-6950-42eb-9ee8-31910479a44c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:07.796677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 692/1410] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:07 2022] POST /identity/v3/users => generated 310 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:07.796677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:07.848500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3541af5-7850-48ff-9ef6-b40c522c2eb1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:07.849498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddae0cb1d13b4d4bbbcb2ba46a39f6d0', 'request_id': 'req-4abdc575-6950-42eb-9ee8-31910479a44c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:07.868347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] The token cache is being invalidate because project ddae0cb1d13b4d4bbbcb2ba46a39f6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:07.868938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:07.876872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4abdc575-6950-42eb-9ee8-31910479a44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:07.876872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 719/1411] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:07 2022] DELETE /identity/v3/projects/ddae0cb1d13b4d4bbbcb2ba46a39f6d0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:07.881415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379e8-15ab-49a4-845c-be54f67cb87b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:07.889441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379e8-15ab-49a4-845c-be54f67cb87b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:07.890714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f379e8-15ab-49a4-845c-be54f67cb87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:07.983538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-96f379e8-15ab-49a4-845c-be54f67cb87b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:07.988671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-96f379e8-15ab-49a4-845c-be54f67cb87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:08.069103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.077837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.077837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.077837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.077837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.077837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.106033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 720/1412] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:08.108391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee411-83b2-4d86-a070-b404942b341d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.109359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee411-83b2-4d86-a070-b404942b341d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.110853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee411-83b2-4d86-a070-b404942b341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.120350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 693/1413] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:08.122187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.124479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:08.142862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4197eb-0796-45be-9f6f-ec97bb756a3c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.152208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.154058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.154742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.155686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] PATH_INFO: `/v3/projects/d555cc0b47f841198e3a21867df20a64/users/04f4910fb1b94b99ab10dd1670a0e59d/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.163879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2faee411-83b2-4d86-a070-b404942b341d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:08.167597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2faee411-83b2-4d86-a070-b404942b341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:08.179853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 721/1414] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:08.186330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faee411-83b2-4d86-a070-b404942b341d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.201207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d555cc0b47f841198e3a21867df20a64, user_id=04f4910fb1b94b99ab10dd1670a0e59d, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.203927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.207739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faee411-83b2-4d86-a070-b404942b341d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.228182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.229453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.234203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.234203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.236684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:08.237252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.243652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.288073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 722/1415] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:08.298803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.309981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f40fe-24ae-4f21-8011-bd347202dac1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.320040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 694/1416] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:08 2022] PUT /identity/v3/projects/d555cc0b47f841198e3a21867df20a64/users/04f4910fb1b94b99ab10dd1670a0e59d/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:08.341676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.345080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.345726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.349484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.349484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.349484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.367434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.491394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:08.502356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2907405-e509-4d9c-8fd9-e1dd1390856b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.505925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 723/1417] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:08 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:08.509080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.596906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:08.628195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.629124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.629492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.629804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.630436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.632239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.646457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 695/1418] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:08.650543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:08.682014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bec2ed-b2f9-420a-aa44-c778abe8a36e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.697457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.702034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.702844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.704320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] PATH_INFO: `/v3/projects/d555cc0b47f841198e3a21867df20a64/users/04f4910fb1b94b99ab10dd1670a0e59d/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.709789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d555cc0b47f841198e3a21867df20a64, user_id=04f4910fb1b94b99ab10dd1670a0e59d, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.714889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e158b-dc02-40cd-b6c2-1eed7863d52e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.736605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 696/1419] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:08 2022] PUT /identity/v3/projects/d555cc0b47f841198e3a21867df20a64/users/04f4910fb1b94b99ab10dd1670a0e59d/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:08.746995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.747593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.748074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.801517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691791-89c2-433d-a171-c58a9860778e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:08.802070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] MFA Rules not processed for user `04f4910fb1b94b99ab10dd1670a0e59d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:08.806435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:08.947674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:08.951428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:08.953102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:08.953400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:08.954081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:08.964481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:08.999572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 697/1420] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:09.001544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.010576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:09.027637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad643e-084b-4725-8f24-9f8838881894 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.033593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.034370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.034680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.035004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] PATH_INFO: `/v3/users/e0d6342874be479daf0b611b71b50ab8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.036219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d6342874be479daf0b611b71b50ab8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.038373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.111173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 724/1421] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:08 2022] POST /identity/v3/users => generated 286 bytes in 602 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:09.113208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.115990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:09.281125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2133-3176-4e42-b155-14798f8cce96 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.285357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.286001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.286317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.286687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.290300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d6342874be479daf0b611b71b50ab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:09.291714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d6342874be479daf0b611b71b50ab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:09.301662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.303390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.312748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d6342874be479daf0b611b71b50ab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:09.335403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 725/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:09.340462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.341004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d6342874be479daf0b611b71b50ab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:09.379971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a59fa-870c-4602-a84a-1afa02e92a3f service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.398545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.399649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.400375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.401116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.402064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.410540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.421376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cf67542d-97f9-4cd4-acfd-97457ccf066b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d6342874be479daf0b611b71b50ab8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:09.432827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 726/1423] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:09.436451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 698/1424] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:08 2022] DELETE /identity/v3/users/e0d6342874be479daf0b611b71b50ab8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:09.439580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.444687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:09.445886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.446187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.446466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.488970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5b0ee-8f01-44ca-8883-62d33f851c46 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.517486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:09.519370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:09.557460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:09.648358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.649102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.649497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.649767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.651296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.657534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.672970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 727/1425] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:09.674271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.676803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:09.692485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccecb-ade8-491e-8ab6-7d4028023195 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.693677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 699/1426] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:09.695574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.702855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.703679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.704044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.704468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] PATH_INFO: `/v3/users/c4f083c1909243e89b719378605d8737` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.705835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f083c1909243e89b719378605d8737)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.708943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.727770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a7e9-ab6b-4b34-82d6-f4668a9744d2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:09.734478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:09.735313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:09.735631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:09.736193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] PATH_INFO: `/v3/projects/321a78273dae423ab2eca0f5045d609e/users/3a8b56ac71a14edcbb717b2578aa4725/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:09.759966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321a78273dae423ab2eca0f5045d609e, user_id=3a8b56ac71a14edcbb717b2578aa4725, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:09.764098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:09.879682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 700/1427] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:09 2022] PUT /identity/v3/projects/321a78273dae423ab2eca0f5045d609e/users/3a8b56ac71a14edcbb717b2578aa4725/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:09.883397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:09.884106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:09.885252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.014676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb25125-b035-4847-81f0-d4dcd2ee6de5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:10.021553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:10.022113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.022438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.022784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.022893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f083c1909243e89b719378605d8737'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:10.067197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f083c1909243e89b719378605d8737'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:10.094390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f083c1909243e89b719378605d8737'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:10.095641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f083c1909243e89b719378605d8737'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:10.142163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:10.143922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e8348-334a-42f2-a084-e3804c4d3569 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:10.156763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 701/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:10.158051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf389-414f-4543-8962-13613a376301 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.158432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf389-414f-4543-8962-13613a376301 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.158798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edf389-414f-4543-8962-13613a376301 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.160622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 702/1429] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:35:10.161913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4b0b-e08b-41ae-9d55-396ed21e5be2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.162301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4b0b-e08b-41ae-9d55-396ed21e5be2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.162667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4b0b-e08b-41ae-9d55-396ed21e5be2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.164248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 703/1430] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:35:10.165462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcd0e-aaf5-40f7-acc3-7e23bfc9f46d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.165805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcd0e-aaf5-40f7-acc3-7e23bfc9f46d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.166146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dcd0e-aaf5-40f7-acc3-7e23bfc9f46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.203555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f083c1909243e89b719378605d8737'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:10.213187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 728/1431] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:09 2022] DELETE /identity/v3/users/c4f083c1909243e89b719378605d8737 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:10.214833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:10.220187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.222245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f82dcd0e-aaf5-40f7-acc3-7e23bfc9f46d None None] MFA Rules not processed for user `3a8b56ac71a14edcbb717b2578aa4725`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:10.223793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f82dcd0e-aaf5-40f7-acc3-7e23bfc9f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.345198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1b01-fdbc-4008-b71c-117412fc4e94 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:10.357605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:10.358217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.358593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.358873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.365120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.382539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 704/1432] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:10.386424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b970d-3058-4311-8fd1-5adb5844d06c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.386424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b970d-3058-4311-8fd1-5adb5844d06c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.386424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b970d-3058-4311-8fd1-5adb5844d06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.447232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fc1b970d-3058-4311-8fd1-5adb5844d06c None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:10.454491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc1b970d-3058-4311-8fd1-5adb5844d06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.488839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:10.490757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b8d0-922f-4bf6-a375-d869a3f1102b service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:10.517211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 729/1433] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:10 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:10.522490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.528714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.528714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.581447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:10.585313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.674673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 705/1434] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:10.676680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.678876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.679327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.754230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:10.755660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.758822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 730/1435] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:10.766832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:10.774736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.795422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 706/1436] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:10.803904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae12c3-2a09-4152-96ed-243264d5c0a9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:10.817059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:10.817730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:10.818075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:10.818359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:10.820691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:10.826082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:10.875298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:10.876813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:10.981512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 731/1437] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:10 2022] POST /identity/v3/projects => generated 371 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:10.986094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:10.994901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:11.064429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c480bc-1f6f-49c9-b03c-8a2eb4cc3f55 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.167005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6538a9d-1a73-474c-ba13-0cf0c9b39b84 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.177434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.178793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.179371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.183084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.194237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.195571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.195934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.196289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.196916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.198998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.219636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 732/1438] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:11.220745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:11.375789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f925ce1-0e61-4c89-bac9-d14b6ff24325 service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.382475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.384069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.384069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.384069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.385085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.389150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.410572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:11.451895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.451895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.480067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 707/1439] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:11.480947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:11.516456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43605-bf11-44e1-b835-f88a6da3f625 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.524373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.528353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.528874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.529342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.549582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.551752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.578852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 708/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:11.609669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 733/1441] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:11 2022] POST /identity/v3/users => generated 292 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:11.613114 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:11.613726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:11.616009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:11.650853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d8b7-e608-4fbb-85be-a8d4d12acb0a service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.671167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.672544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93ffc-7d8d-4e50-9756-d62d7f0e8cef admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.676493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.677151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.677849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] PATH_INFO: `/v3/projects/6ede276d10e247528cfda3008983bf6c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.689850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ede276d10e247528cfda3008983bf6c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.697238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.720603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.728821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.730118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.734148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.734148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.736039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.822790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 734/1442] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:11.828738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:11.858001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2602b6e-d312-439d-8d6d-47905082a496 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:11.865908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:11.867380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:11.867974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:11.883589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] PATH_INFO: `/v3/projects/c5e7affb94a94957924c3a3109dea34f/users/aa5bcc12effa404bac844b65d2a48fc2/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:11.928473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e7affb94a94957924c3a3109dea34f, user_id=aa5bcc12effa404bac844b65d2a48fc2, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:11.936857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:11.938847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ede276d10e247528cfda3008983bf6c', 'request_id': 'req-4d6794de-fdca-42e5-be25-81ac7043dbd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:11.962947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ede276d10e247528cfda3008983bf6c', 'request_id': 'req-4d6794de-fdca-42e5-be25-81ac7043dbd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.046771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 735/1443] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:11 2022] PUT /identity/v3/projects/c5e7affb94a94957924c3a3109dea34f/users/aa5bcc12effa404bac844b65d2a48fc2/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:12.050495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:12.066347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ede276d10e247528cfda3008983bf6c', 'request_id': 'req-4d6794de-fdca-42e5-be25-81ac7043dbd3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.099463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] The token cache is being invalidate because project 6ede276d10e247528cfda3008983bf6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:12.099951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.106380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.114404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 709/1444] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:11 2022] DELETE /identity/v3/projects/6ede276d10e247528cfda3008983bf6c => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:12.115506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:12.123859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:12.157643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:12.201444 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d7454-7fd5-47da-8022-5ed1252d4c04 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:12.206552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:12.207175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.207512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.207792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.209258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:12.211061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:12.225418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 736/1445] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:12.229340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:12.262140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:12.289081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99c47a-1058-4fcb-9dfe-707a595bed89 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:12.300793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:12.301516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.301787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.302070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] PATH_INFO: `/v3/projects/c5e7affb94a94957924c3a3109dea34f/users/aa5bcc12effa404bac844b65d2a48fc2/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.304559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e7affb94a94957924c3a3109dea34f, user_id=aa5bcc12effa404bac844b65d2a48fc2, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:12.307454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b0ad5-8e24-47d2-b1eb-c9fd9f2da2f8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:12.341060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 737/1446] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:12 2022] PUT /identity/v3/projects/c5e7affb94a94957924c3a3109dea34f/users/aa5bcc12effa404bac844b65d2a48fc2/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:12.341270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6794de-fdca-42e5-be25-81ac7043dbd3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:12.343249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.343537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.343818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.406079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] MFA Rules not processed for user `aa5bcc12effa404bac844b65d2a48fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:12.407407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:12.536697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:12.537386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.538583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.538583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] PATH_INFO: `/v3/users/486ef9292abe4ecaa41190711951a846` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.539611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=486ef9292abe4ecaa41190711951a846)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:12.541568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:12.592442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 738/1447] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:12.604132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:12.605582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:12.697929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826d5c0-43da-49c4-8be5-bd846d42b6a5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:12.715946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:12.716629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.716937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.718327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.733659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:12.735480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:12.749997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 739/1448] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:12.751353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:12.753138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:12.769275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8d5f-31cd-4815-bc29-404530c98581 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:12.776620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:12.777928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:12.778707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:12.779276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] PATH_INFO: `/v3/projects/8f5b23f416f54e5ca4763ba2b6562ed8` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:12.781476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5b23f416f54e5ca4763ba2b6562ed8)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:12.788662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:12.867895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486ef9292abe4ecaa41190711951a846'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.872665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486ef9292abe4ecaa41190711951a846'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.917176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486ef9292abe4ecaa41190711951a846'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.953356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486ef9292abe4ecaa41190711951a846'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.957971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5b23f416f54e5ca4763ba2b6562ed8', 'request_id': 'req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:12.965369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5b23f416f54e5ca4763ba2b6562ed8', 'request_id': 'req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:13.110886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5b23f416f54e5ca4763ba2b6562ed8', 'request_id': 'req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:13.128727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] The token cache is being invalidate because project 8f5b23f416f54e5ca4763ba2b6562ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:13.129059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:13.129830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d31f1c8a-7b65-44aa-b13e-89957a4b26c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:13.134606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 740/1449] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:12 2022] DELETE /identity/v3/projects/8f5b23f416f54e5ca4763ba2b6562ed8 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:13.146734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.147401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.147906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.191458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-20284063-bef8-4f19-810d-3386fa10f1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486ef9292abe4ecaa41190711951a846'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:13.196543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:13.198212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:13.202021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 710/1450] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:12 2022] DELETE /identity/v3/users/486ef9292abe4ecaa41190711951a846 => generated 0 bytes in 1087 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:13.303492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 741/1451] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:13.311391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.312291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.312804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.363838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:13.366155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:13.380371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 711/1452] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:13.384160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:13.400795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3e3a7-58a8-482c-85af-13d12a2597de None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:13.415261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:13.416164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.416283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.416480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.417173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:13.417477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:13.419378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:13.448085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 742/1453] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:13.453438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:13.466680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633e87e-a3e9-4387-9947-081ceb6a4dfb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:13.479906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:13.481352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.481352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.481352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.482333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:13.484997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:13.616995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 712/1454] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:13 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:13.625042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:13.642769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e5ad1-ff08-4d79-bd42-3811f49e0b39 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:13.761028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:13.761028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.761028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.761028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.761601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:13.763142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:13.882660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 743/1455] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:13 2022] POST /identity/v3/users => generated 281 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:13.886762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:13.901658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8e36-d853-4a9a-bfed-bfcc943b91a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:13.918004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:13.918720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:13.919040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:13.919317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:13.919999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:13.921604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:13.959599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 713/1456] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:13.959817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:13.985111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c666fc1-7957-4b00-a3ab-201f960fdea2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:14.001730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:14.001730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.001947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:14.002206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:14.003083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:14.005035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c08f4-4568-4130-85af-5e8ae24e3819 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:14.026882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 744/1457] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:14.031681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:14.046307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b5c5-b469-4186-9085-dc74cbc7858a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:14.068407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:14.070614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.071083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:14.073402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] PATH_INFO: `/v3/projects/25de4d4cadff438b82efd4304e9d6fbc/users/a4c33a64c0f94eee880fc635f49663a0/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:14.125956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25de4d4cadff438b82efd4304e9d6fbc, user_id=a4c33a64c0f94eee880fc635f49663a0, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:14.132763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:14.287050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 714/1458] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:14 2022] PUT /identity/v3/projects/25de4d4cadff438b82efd4304e9d6fbc/users/a4c33a64c0f94eee880fc635f49663a0/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:14.302641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.303116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:14.303435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:14.322737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:14.324800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:14.327714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:14.400042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] MFA Rules not processed for user `a4c33a64c0f94eee880fc635f49663a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:14.406170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:14.480795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af25ff-4218-47dd-a18a-d36babd08f6b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:14.499806 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:14.500780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.503321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:14.503321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] PATH_INFO: `/v3/users/3a8b56ac71a14edcbb717b2578aa4725` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:14.503321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8b56ac71a14edcbb717b2578aa4725)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:14.507382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:14.563685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 745/1459] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:14.568788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:14.608052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8afdd-c5ee-40ff-b417-8cc477c3fb13 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:14.616995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:14.624217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:14.625270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.625581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:14.625858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:14.626546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:14.636954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:14.763422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8b56ac71a14edcbb717b2578aa4725'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:14.764638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8b56ac71a14edcbb717b2578aa4725'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:14.800846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8b56ac71a14edcbb717b2578aa4725'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:14.815460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 746/1460] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:14 2022] POST /identity/v3/projects => generated 363 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:14.824394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:14.828819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:14.849924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8b56ac71a14edcbb717b2578aa4725'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:14.968664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f09fced4-8577-4ebf-b3df-924da759279b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8b56ac71a14edcbb717b2578aa4725'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:14.978009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 715/1461] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:14 2022] DELETE /identity/v3/users/3a8b56ac71a14edcbb717b2578aa4725 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:14.983073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:14.998026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.007653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.056327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e550c-4286-4f96-afed-4db5b7c2c48c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.057897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:15.060611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:15.162852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 716/1462] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:15.172518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.174737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:15.233994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.235336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.235620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.235932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.236800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.239072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.275576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5523593-9a1c-4e08-8dfa-926c8b9c12e8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.280053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.280833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.281176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.281549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.296775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.299362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.311145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 717/1463] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:15.342416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:15.400342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 747/1464] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:14 2022] POST /identity/v3/users => generated 288 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:15.404431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.422058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4eada-0ec8-4845-b4c1-e1ece9be8445 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.428097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.428770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.429313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.429609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.430263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.431953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.447269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 718/1465] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:15.452007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.478184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127d264-2319-4bcd-bb18-5af220a6cd80 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.480423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.481160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.482053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.482481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] PATH_INFO: `/v3/projects/a26d4b4a9b914da68923ee9d7e47366c/users/90407be7382446b3a8909ba4d32fea3a/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.504137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d4b4a9b914da68923ee9d7e47366c, user_id=90407be7382446b3a8909ba4d32fea3a, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.510587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.543463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.547154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:15.565650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41b36-4b01-40b9-b426-8bcad7cae11a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.571065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.572044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.572506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.573078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] PATH_INFO: `/v3/projects/668a5402ab024c638b2512af9572d6bf` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.574729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668a5402ab024c638b2512af9572d6bf)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.577608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.614171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 748/1466] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:15 2022] PUT /identity/v3/projects/a26d4b4a9b914da68923ee9d7e47366c/users/90407be7382446b3a8909ba4d32fea3a/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:15.617923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.711663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae5388-2320-4eaf-bebb-ee0ca9314424 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.717004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.717643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.718501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.718501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.719476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.721273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.732936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 749/1467] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:15.745198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:15.759080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ba874-cc35-4697-8515-b25939ad9851 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:15.776711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668a5402ab024c638b2512af9572d6bf', 'request_id': 'req-81d71c1a-008c-4fb3-8d6f-c08184d13261'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:15.788331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668a5402ab024c638b2512af9572d6bf', 'request_id': 'req-81d71c1a-008c-4fb3-8d6f-c08184d13261'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:15.883545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668a5402ab024c638b2512af9572d6bf', 'request_id': 'req-81d71c1a-008c-4fb3-8d6f-c08184d13261'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:15.890565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:15.891412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.891767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.892102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] PATH_INFO: `/v3/projects/a26d4b4a9b914da68923ee9d7e47366c/users/90407be7382446b3a8909ba4d32fea3a/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.898486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d4b4a9b914da68923ee9d7e47366c, user_id=90407be7382446b3a8909ba4d32fea3a, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:15.901380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4521fa-7458-46b5-8689-9ca29f2f42d5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:15.916954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 750/1468] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:15 2022] PUT /identity/v3/projects/a26d4b4a9b914da68923ee9d7e47366c/users/90407be7382446b3a8909ba4d32fea3a/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:15.917681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] The token cache is being invalidate because project 668a5402ab024c638b2512af9572d6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:15.917969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:15.920691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-81d71c1a-008c-4fb3-8d6f-c08184d13261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:15.922392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 719/1469] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:15 2022] DELETE /identity/v3/projects/668a5402ab024c638b2512af9572d6bf => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:15.926683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.927026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.927365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.931786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efbdc3-630b-4cbe-8527-e2fcb438cc9a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:15.933286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efbdc3-630b-4cbe-8527-e2fcb438cc9a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:15.935488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efbdc3-630b-4cbe-8527-e2fcb438cc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:15.978993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] MFA Rules not processed for user `90407be7382446b3a8909ba4d32fea3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:15.979286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a9efbdc3-630b-4cbe-8527-e2fcb438cc9a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:15.980483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:15.980768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a9efbdc3-630b-4cbe-8527-e2fcb438cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.058224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 720/1470] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:16.066918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 751/1471] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:16.071575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.071575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.071575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.092325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.093854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.106626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:16.109231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.126971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 721/1472] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:16.130599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.140504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51937fa-8a04-44f8-b259-188a14d8ff4b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.145695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.146245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.146577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.146988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.147654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:16.147963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.149618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.162979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5aedd-a79f-46f2-9b62-627669bedb6a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.163580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 722/1473] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:35:16.167258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.167786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.168019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.168380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.168757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.178797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.179740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742e567-eeaa-4f4e-8e6f-20036d71ba23 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.180552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.184731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.185631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.186019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.189756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.189756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.189756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.192675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 752/1474] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:16.211696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:16.246435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.248416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.268230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a281c-0818-4fe5-bb22-ccb5039a495b service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.283538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.284264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.284550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.284854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] PATH_INFO: `/v3/projects/321a78273dae423ab2eca0f5045d609e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.286020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321a78273dae423ab2eca0f5045d609e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.288255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.293912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 723/1475] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:16 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:16.298206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.363278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a40c6-7c42-4126-a93c-c9877ee9eb78 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.368032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.368641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.368928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.369201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.369847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.371830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.438788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321a78273dae423ab2eca0f5045d609e', 'request_id': 'req-6b5ecab6-3812-45ac-aacd-ce11c0045838'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.444408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321a78273dae423ab2eca0f5045d609e', 'request_id': 'req-6b5ecab6-3812-45ac-aacd-ce11c0045838'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.456176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 724/1476] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:16 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:16.457164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.529211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3894e-8fab-4016-846c-9b6eaa75ee81 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.540100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321a78273dae423ab2eca0f5045d609e', 'request_id': 'req-6b5ecab6-3812-45ac-aacd-ce11c0045838'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.563881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] The token cache is being invalidate because project 321a78273dae423ab2eca0f5045d609e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:16.564166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.564993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.567711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 753/1477] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:16 2022] DELETE /identity/v3/projects/321a78273dae423ab2eca0f5045d609e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:16.567711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.568902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.588027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.588790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.589181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.589693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.591754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.645757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ecab6-3812-45ac-aacd-ce11c0045838 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.651749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.652523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.652813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.653150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.653839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.656033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.668467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.670322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 754/1478] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:16.670710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.672053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.674299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.681382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 725/1479] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:16.682613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.693322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d820ade-f52f-4074-a5c4-a05454abbbf1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.705888 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d99df-5bdd-4c88-b0c5-4bbdf935d821 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.708985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.709698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.710900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.711260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] PATH_INFO: `/v3/users/aa5bcc12effa404bac844b65d2a48fc2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.711387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.712071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.712427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.712643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5bcc12effa404bac844b65d2a48fc2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.712840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.713648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.714664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.715465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.729073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 726/1480] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:16.733180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.743516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce0878-166e-44b5-8949-f6e973bd2282 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:16.748359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:16.749010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.749293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.749568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] PATH_INFO: `/v3/projects/a97d1e92dcb94b9f9fca17455c735a2f/users/780dc238353344dc9711e1d91ba2eafc/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.764293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97d1e92dcb94b9f9fca17455c735a2f, user_id=780dc238353344dc9711e1d91ba2eafc, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:16.767081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12944b51-8096-4da6-a5e1-8f82f48ad6fd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:16.858421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 727/1481] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:16 2022] PUT /identity/v3/projects/a97d1e92dcb94b9f9fca17455c735a2f/users/780dc238353344dc9711e1d91ba2eafc/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:16.868517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:16.868807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:16.869083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:16.895855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] MFA Rules not processed for user `780dc238353344dc9711e1d91ba2eafc`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:16.897118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:16.921384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5bcc12effa404bac844b65d2a48fc2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.928652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5bcc12effa404bac844b65d2a48fc2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.940879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5bcc12effa404bac844b65d2a48fc2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.941618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5bcc12effa404bac844b65d2a48fc2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:16.962554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:16.971732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 728/1482] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:16.971732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:16.972182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.020989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:17.021739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5bcc12effa404bac844b65d2a48fc2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.028304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 755/1483] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:16 2022] DELETE /identity/v3/users/aa5bcc12effa404bac844b65d2a48fc2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:17.036007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab9f85-e630-4a22-93ae-9c6fcafb5945 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.095566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.096397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.096686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.096964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.097608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.100644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.203947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 729/1484] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:16 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:17.205331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.215529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb76788-7b76-42e3-865d-1c4cf4207f8a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.283196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.284271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.284271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.284621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.285367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.287426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.406486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 756/1485] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:17 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:17.407718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.420117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919f659-3190-459e-922a-189aa030b37f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.426167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.427423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.427935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.428200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.428887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.430479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.442698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 730/1486] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:17.447133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.458274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e10cb-587c-49cb-a7a8-f9e66253bb1a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.463284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.464051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.464404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.464817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] PATH_INFO: `/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d/users/9dac03e5e97047bfbd903bc7e84d0376/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.480467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c53627a860f498d8b6d1877ec6e5d2d, user_id=9dac03e5e97047bfbd903bc7e84d0376, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.483785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.541919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.543070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 757/1487] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:17 2022] PUT /identity/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d/users/9dac03e5e97047bfbd903bc7e84d0376/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:17.544056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.546921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.558907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8373-570c-4295-a3c8-a9b80bc40400 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.612198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441719db-2532-49cd-bbd1-b215d5a23b43 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.616869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.617478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.617760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.618034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.618387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.618704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.619054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.619340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.619665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] PATH_INFO: `/v3/projects/c5e7affb94a94957924c3a3109dea34f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.620451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.620895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e7affb94a94957924c3a3109dea34f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.622833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.630588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 758/1488] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:17.634821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.644882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc76d8-e26d-4839-8906-0da589d3db49 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.649643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.650308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.650618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.650892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] PATH_INFO: `/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d/users/9dac03e5e97047bfbd903bc7e84d0376/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.652803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c53627a860f498d8b6d1877ec6e5d2d, user_id=9dac03e5e97047bfbd903bc7e84d0376, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.656059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f165-dceb-4a1f-a817-73b4382647de admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.666164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 759/1489] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:17 2022] PUT /identity/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d/users/9dac03e5e97047bfbd903bc7e84d0376/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:17.676941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.684317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.684317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.714471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] MFA Rules not processed for user `9dac03e5e97047bfbd903bc7e84d0376`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:17.714471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.720245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e7affb94a94957924c3a3109dea34f', 'request_id': 'req-d5271d7a-dc99-4b26-bf24-e7326da0cee5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.727333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e7affb94a94957924c3a3109dea34f', 'request_id': 'req-d5271d7a-dc99-4b26-bf24-e7326da0cee5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.784577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e7affb94a94957924c3a3109dea34f', 'request_id': 'req-d5271d7a-dc99-4b26-bf24-e7326da0cee5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.789305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 760/1490] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:17.797713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] The token cache is being invalidate because project c5e7affb94a94957924c3a3109dea34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:17.798011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.801472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.802971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.803470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d5271d7a-dc99-4b26-bf24-e7326da0cee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:17.805203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 731/1491] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:17 2022] DELETE /identity/v3/projects/c5e7affb94a94957924c3a3109dea34f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:17.810044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.810335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.810670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.835519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:17.836838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.873905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8421f1-a3fb-4fdc-b307-d82043879e4b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.877654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.878244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.878540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.878815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.890748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.893264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.895128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a4293-08d9-4d7d-b76a-42d7533e3220 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.904812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 761/1492] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:17.910559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 732/1493] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:17.921030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.921430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.921708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.942635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:17.943838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:17.952541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 762/1494] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:17.956854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:17.964137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:17.967904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b455c-86c4-4709-bd89-2334a71a12e2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:17.972477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:17.973082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:17.973563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:17.973675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:17.974279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:17.974611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:17.976204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:17.988113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 733/1495] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:17.992455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.002642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e107-33e9-4c37-a632-1370cc5f41fc None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.007599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.008212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.008493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.008765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.009410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.019195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.037820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:18.101009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 763/1496] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:17 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:18.101550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.112338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5593f-8662-48f6-b968-e939622e8786 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.171327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.171939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.172218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.172491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.176164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.176164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.294643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 734/1497] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:18 2022] POST /identity/v3/users => generated 282 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:18.298763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.309177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7e87-021c-4d13-a232-d235dfa9dd00 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.315434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.316057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.316337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.316610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.317262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.318910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.330462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 764/1498] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:18.334310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.345915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332ae0-b818-4606-b98b-d2a822b25c62 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.351470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.352077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.352358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.352632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.353286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.354963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8dfbd-5c01-4793-8e08-0bd2f231100e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.366236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 735/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:18.370568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.380915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d03b-cdc8-4d93-bff6-4f00c0b5ca31 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.385533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.386299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.386630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.386908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] PATH_INFO: `/v3/projects/b5bd891b4aaf427dbd0865cfd9de3039/users/bd39a8328b604bfebc71319433a5bdb6/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.405587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bd891b4aaf427dbd0865cfd9de3039, user_id=bd39a8328b604bfebc71319433a5bdb6, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.408395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.512153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 765/1500] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:18 2022] PUT /identity/v3/projects/b5bd891b4aaf427dbd0865cfd9de3039/users/bd39a8328b604bfebc71319433a5bdb6/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:18.522987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.523428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.523516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.552639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] MFA Rules not processed for user `bd39a8328b604bfebc71319433a5bdb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:18.553847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:18.622847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 736/1501] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:18.627420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.692207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:18.701104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7bbb2-3c31-46b7-9f8d-0d5e3af05162 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.706854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.718680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.718680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.718680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.718680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.718680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.815030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 766/1502] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:18 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:18.816162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.828069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eea2cd-98e2-44ed-8b87-463f5d1b986c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:18.884648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:18.885287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:18.885572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:18.885877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:18.886535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:18.888496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:18.983279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 737/1503] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:18 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:18.985254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:18.995869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c6466-1819-4dd3-a28c-dd9155bd2843 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:19.003427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:19.008129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:19.008511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.008860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.009136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.010543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:19.012195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:19.027375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 767/1504] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:19.035256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:19.053658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5730ae-189e-4c92-b265-d4e12bf00438 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:19.058572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:19.082781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.082781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.082781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] PATH_INFO: `/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74/users/a83a93f82f65446db46c05370c39b858/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.082781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44d2f228b3d4c0ea98c7dadfd40be74, user_id=a83a93f82f65446db46c05370c39b858, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:19.082781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:19.151551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 738/1505] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:19 2022] PUT /identity/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74/users/a83a93f82f65446db46c05370c39b858/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:19.156212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:19.166252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef6590-4b91-4424-ad44-c96323fb1ffb admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:19.232848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:19.233511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.233744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.234016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.234707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:19.236298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28998beb-aa77-4329-a32c-753a1478c191 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:19.246025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 768/1506] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:19.252438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:19.263187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8a9ba-7971-4e11-b309-c3b572e61963 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:19.268131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:19.268895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.269217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.269491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] PATH_INFO: `/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74/users/a83a93f82f65446db46c05370c39b858/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.271520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44d2f228b3d4c0ea98c7dadfd40be74, user_id=a83a93f82f65446db46c05370c39b858, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:19.274167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:19.284906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 739/1507] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:19 2022] PUT /identity/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74/users/a83a93f82f65446db46c05370c39b858/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:19.295172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.295505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.295781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.326325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] MFA Rules not processed for user `a83a93f82f65446db46c05370c39b858`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:19.327627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:19.346132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:19.396948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 769/1508] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:19.408385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:19.410109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:19.471412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c17e7-002b-45aa-bf39-4550b2d4bab5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:19.475065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:19.475969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:19.475969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:19.476237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:19.485666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:19.487099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:19.496693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 740/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:19.632317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:19.740310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:19.786609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:19.963638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:20.077676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:20.609297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:20.690875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef546cd-b6c8-452a-bf8c-5db048a7d979 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:20.697733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:20.698819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:20.699418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:20.699697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] PATH_INFO: `/v3/users/bd39a8328b604bfebc71319433a5bdb6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:20.701431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd39a8328b604bfebc71319433a5bdb6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:20.704080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:20.836569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:20.838843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:20.840280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:20.855568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35bb97-17fe-492a-affa-91cda6e1b792 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:20.860772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:20.861470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:20.861753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:20.862036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] PATH_INFO: `/v3/users/780dc238353344dc9711e1d91ba2eafc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:20.863380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780dc238353344dc9711e1d91ba2eafc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:20.865646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:20.900641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:20.958886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd39a8328b604bfebc71319433a5bdb6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:20.972925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd39a8328b604bfebc71319433a5bdb6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:20.995744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd39a8328b604bfebc71319433a5bdb6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:20.998704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd39a8328b604bfebc71319433a5bdb6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.109145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e615c9-fa07-4ec6-b2de-e496d7c12b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd39a8328b604bfebc71319433a5bdb6'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.118278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 770/1510] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:20 2022] DELETE /identity/v3/users/bd39a8328b604bfebc71319433a5bdb6 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:21.119656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:21.119995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:21.120350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:21.146108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:21.147476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:21.193873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780dc238353344dc9711e1d91ba2eafc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.194918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780dc238353344dc9711e1d91ba2eafc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.207046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780dc238353344dc9711e1d91ba2eafc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.226525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780dc238353344dc9711e1d91ba2eafc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.240215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 771/1511] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:21.251333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:21.253929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:21.336132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3fd03-d969-410f-91ad-968a69a623bd None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:21.336132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:21.336132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:21.336688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:21.337051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:21.347931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:21.350123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:21.365840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-774862f9-2018-447d-9b55-1a4bc526edd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780dc238353344dc9711e1d91ba2eafc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:21.369250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 741/1512] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:20 2022] DELETE /identity/v3/users/780dc238353344dc9711e1d91ba2eafc => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:21.373973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:21.374339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:21.374780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:21.391936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 772/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:21.408422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:21.422711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:21.530907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 742/1514] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:21.531078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:21.531078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:21.540907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7956-2f8f-4e31-8923-afcb562711b4 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:21.608419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:21.609139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:21.609578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:21.610015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:21.621647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:21.623215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:21.633603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 773/1515] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:21.827239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:21.972121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:21.978716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:21.993401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecea57-12b4-42b7-8006-81aee3540fd7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:22.017343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:22.018022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:22.018315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:22.018627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] PATH_INFO: `/v3/projects/b5bd891b4aaf427dbd0865cfd9de3039` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:22.020162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bd891b4aaf427dbd0865cfd9de3039)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:22.022726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:22.053980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:22.190899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bd891b4aaf427dbd0865cfd9de3039', 'request_id': 'req-6570e8a8-4c6c-47e6-9991-40b46ddae766'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.209244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:22.214572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:22.226947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bd891b4aaf427dbd0865cfd9de3039', 'request_id': 'req-6570e8a8-4c6c-47e6-9991-40b46ddae766'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.241236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c6506-30fb-4a96-a296-2fd24669df8a service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:22.316459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:22.317249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:22.317535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:22.317811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] PATH_INFO: `/v3/projects/a97d1e92dcb94b9f9fca17455c735a2f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:22.319026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97d1e92dcb94b9f9fca17455c735a2f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:22.321079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:22.327483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bd891b4aaf427dbd0865cfd9de3039', 'request_id': 'req-6570e8a8-4c6c-47e6-9991-40b46ddae766'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.339929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] The token cache is being invalidate because project b5bd891b4aaf427dbd0865cfd9de3039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:22.340301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.341766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.343385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 743/1516] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:21 2022] DELETE /identity/v3/projects/b5bd891b4aaf427dbd0865cfd9de3039 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:22.347302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:22.348753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:22.362418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570e8a8-4c6c-47e6-9991-40b46ddae766 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:22.370652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:22.370652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:22.370652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:22.370652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] PATH_INFO: `/v3/users/a83a93f82f65446db46c05370c39b858` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:22.371044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83a93f82f65446db46c05370c39b858)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:22.371537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:22.460189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97d1e92dcb94b9f9fca17455c735a2f', 'request_id': 'req-5113f559-ed52-44ac-8910-c73c39c344c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.466480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97d1e92dcb94b9f9fca17455c735a2f', 'request_id': 'req-5113f559-ed52-44ac-8910-c73c39c344c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.529560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97d1e92dcb94b9f9fca17455c735a2f', 'request_id': 'req-5113f559-ed52-44ac-8910-c73c39c344c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.543600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] The token cache is being invalidate because project a97d1e92dcb94b9f9fca17455c735a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:22.544138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.544336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83a93f82f65446db46c05370c39b858'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.544973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.545111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83a93f82f65446db46c05370c39b858'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.546590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 774/1517] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:22 2022] DELETE /identity/v3/projects/a97d1e92dcb94b9f9fca17455c735a2f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:22.550445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:22.552197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:22.564784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83a93f82f65446db46c05370c39b858'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.582382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83a93f82f65446db46c05370c39b858'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.624205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f559-ed52-44ac-8910-c73c39c344c0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:22.628788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:22.629421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:22.629816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:22.630100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] PATH_INFO: `/v3/users/9dac03e5e97047bfbd903bc7e84d0376` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:22.631274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dac03e5e97047bfbd903bc7e84d0376)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:22.633190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:22.671441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83a93f82f65446db46c05370c39b858'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.681538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 744/1518] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:22 2022] DELETE /identity/v3/users/a83a93f82f65446db46c05370c39b858 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:22.862225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dac03e5e97047bfbd903bc7e84d0376'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.872251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dac03e5e97047bfbd903bc7e84d0376'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.903884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dac03e5e97047bfbd903bc7e84d0376'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:22.909225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dac03e5e97047bfbd903bc7e84d0376'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.014346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b00e675b-b8c1-4e79-9d9c-a812f57b9210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dac03e5e97047bfbd903bc7e84d0376'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.025816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 775/1519] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:22 2022] DELETE /identity/v3/users/9dac03e5e97047bfbd903bc7e84d0376 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:23.096080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.097644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:23.198296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d187d-c7c9-4b11-a618-1a3c3cc6cf7e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:23.263776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:23.265022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.265743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.266106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] PATH_INFO: `/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.269220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44d2f228b3d4c0ea98c7dadfd40be74)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:23.272033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:23.394617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44d2f228b3d4c0ea98c7dadfd40be74', 'request_id': 'req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.401638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44d2f228b3d4c0ea98c7dadfd40be74', 'request_id': 'req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.478359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44d2f228b3d4c0ea98c7dadfd40be74', 'request_id': 'req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.484179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:23.489124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] The token cache is being invalidate because project f44d2f228b3d4c0ea98c7dadfd40be74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:23.489502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.490374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9e9632b9-e883-48c7-b3c4-a13e5ce4e3b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.493665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 745/1520] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:23 2022] DELETE /identity/v3/projects/f44d2f228b3d4c0ea98c7dadfd40be74 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:23.499356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.499723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.500069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.524712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:23.527787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:23.618495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 776/1521] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:23.627330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.627685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.628027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.648628 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:23.649838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:23.658469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 746/1522] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:23.662948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.674941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609869bb-55d3-4703-9fb2-e0ccfa35ce83 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:23.682595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.684746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:23.690374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:23.691310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.691736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.692200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.695151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:23.696140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:23.699088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:23.699760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9543-8a0f-4268-8d2b-b815e29e192c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:23.706134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:23.707262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.707557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.707833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] PATH_INFO: `/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.709168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c53627a860f498d8b6d1877ec6e5d2d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:23.712029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:23.734553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 777/1523] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:23.736077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.747350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2c5b8-b151-492a-8804-f52cf2cba91c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:23.753141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:23.753751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.754388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.754708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.755535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:23.758194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:23.774667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:23.837604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c53627a860f498d8b6d1877ec6e5d2d', 'request_id': 'req-bf567c3b-052c-405c-a0f3-f04050a8c39d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.843179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c53627a860f498d8b6d1877ec6e5d2d', 'request_id': 'req-bf567c3b-052c-405c-a0f3-f04050a8c39d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.871333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 778/1524] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:23 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:23.871333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.917448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c53627a860f498d8b6d1877ec6e5d2d', 'request_id': 'req-bf567c3b-052c-405c-a0f3-f04050a8c39d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.930313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] The token cache is being invalidate because project 2c53627a860f498d8b6d1877ec6e5d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:23.930665 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.931322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:23.932877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 747/1525] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:23 2022] DELETE /identity/v3/projects/2c53627a860f498d8b6d1877ec6e5d2d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:23.936747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668cab-7909-46c6-9942-698dbba576b6 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:23.937653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:23.941276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:23.941609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:23.942220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:23.942499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:23.942796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:23.943425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:23.945378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa0cd4-8eae-42a5-9243-c9b33d40fde3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.008443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf567c3b-052c-405c-a0f3-f04050a8c39d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.013315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d495-378a-43a7-8108-98053299263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.013964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d495-378a-43a7-8108-98053299263c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.014244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d495-378a-43a7-8108-98053299263c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.014541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d495-378a-43a7-8108-98053299263c admin admin] PATH_INFO: `/v3/users/a4c33a64c0f94eee880fc635f49663a0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.015710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236d495-378a-43a7-8108-98053299263c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c33a64c0f94eee880fc635f49663a0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.017744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236d495-378a-43a7-8108-98053299263c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.035310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 779/1526] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:23 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:24.035310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.035310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.035310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.053485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:24.055398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:24.056639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.065574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 780/1527] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:24.066544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.067718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.079378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17eac91-4e71-47b4-8f5c-6d39db94c470 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.083921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.084513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.084791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.085079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.085700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.087385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7dd876-b979-4277-8c1d-0ab21b1c0b57 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.097506 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 781/1528] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:24.098943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.099306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.099678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.119686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:24.120949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.129407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 782/1529] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:24.130180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.141842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f586f-e825-4b4a-9832-d73ee0363263 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.147508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.148054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.148329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.148606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.149293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.150927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.161556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 783/1530] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:24.162345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.172418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b83f-8e10-40ce-9412-2acd4e9e9e9d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.177330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.177929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.178207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.178479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.179166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:24.179468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.181019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.193613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 784/1531] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:24.194446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.203802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922250-e3d4-48ab-aade-8dd43b1cf07f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.208490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.210133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.210133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.210133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] PATH_INFO: `/v3/projects/5c8ddc1d11ef4c01a5a134b2612bc363/users/871e1c0a885f46c0848ea480f792554a/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.222794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3236d495-378a-43a7-8108-98053299263c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c33a64c0f94eee880fc635f49663a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:24.223535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3236d495-378a-43a7-8108-98053299263c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c33a64c0f94eee880fc635f49663a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:24.228474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c8ddc1d11ef4c01a5a134b2612bc363, user_id=871e1c0a885f46c0848ea480f792554a, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.230923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3236d495-378a-43a7-8108-98053299263c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c33a64c0f94eee880fc635f49663a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:24.231958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.244145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3236d495-378a-43a7-8108-98053299263c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c33a64c0f94eee880fc635f49663a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:24.305815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3236d495-378a-43a7-8108-98053299263c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c33a64c0f94eee880fc635f49663a0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:24.312323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 785/1532] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:24 2022] PUT /identity/v3/projects/5c8ddc1d11ef4c01a5a134b2612bc363/users/871e1c0a885f46c0848ea480f792554a/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:24.312828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 748/1533] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:23 2022] DELETE /identity/v3/users/a4c33a64c0f94eee880fc635f49663a0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:24.315689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.315689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.322672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f06c-6fc1-44f1-9285-edb2d5f64ca2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.322961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f06c-6fc1-44f1-9285-edb2d5f64ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.323263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913f06c-6fc1-44f1-9285-edb2d5f64ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.342644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4913f06c-6fc1-44f1-9285-edb2d5f64ca2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:24.343861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4913f06c-6fc1-44f1-9285-edb2d5f64ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.380601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b55a6-84e6-48f9-a657-171d7e8b8c21 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.407244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 749/1534] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:24.408613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.408901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.409172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.443229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.443848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.444134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.444411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.445051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.447017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.450239 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] MFA Rules not processed for user `871e1c0a885f46c0848ea480f792554a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:24.451486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.520399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 750/1535] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:24.521467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.522590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.547985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 786/1536] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:24.548950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.554242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:24.589980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b9060-7b59-451e-b742-6f19171241f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.631365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdaa85-8710-47d7-b9e6-6641a5c945f7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.637714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.638062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.638386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.639009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.639934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.641924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.646900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.648354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.650260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.650608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:24.651398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.669772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.671550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.682554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 751/1537] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:24.683524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.693291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9431cd8-f113-4330-bb7b-7d94c5462bd3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.698308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.699368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.700485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.700890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.701575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.703624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.742936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 787/1538] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:24.747033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.764531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:24.816595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b68ef-6de4-432e-a6b4-478eed1a566e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.821939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.822575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.822857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.823126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.823760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.826214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.828351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 752/1539] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:24.832685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.843145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e170e1-24f8-407b-9c49-ab9ddf64c1d0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.848266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.852639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.852639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.852639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.852639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.852639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.862759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 753/1540] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:24.866737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.876266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b2bb7-e79d-407d-9cf5-3e209135f5a0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.880938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.881566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.881845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.882119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.882778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.884341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.902216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 754/1541] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:24.906634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.917204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908273a-fef7-49d7-b05e-63ce97287566 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.923972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.925293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.925594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.925897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] PATH_INFO: `/v3/projects/b0e5776637bc4aceb3bccdca0f7e480f/users/33b35fc98a87460caeab99fe50fc35fb/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.936149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 788/1542] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:24 2022] POST /identity/v3/users => generated 300 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:24.937552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:24.951110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e5776637bc4aceb3bccdca0f7e480f, user_id=33b35fc98a87460caeab99fe50fc35fb, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.954645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e9b32-a899-4ce8-bb7f-2b5820ec0273 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.968196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5fe-2655-479b-9d4e-018b60217fd0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:24.976665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:24.977294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:24.977573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:24.977845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:24.978496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:24.980171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:24.993200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 789/1543] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:24.997948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.020937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 755/1544] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:24 2022] PUT /identity/v3/projects/b0e5776637bc4aceb3bccdca0f7e480f/users/33b35fc98a87460caeab99fe50fc35fb/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:25.022656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0493e-f11a-4886-b340-0089ad585674 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.027353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.027719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.028095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.062257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] MFA Rules not processed for user `33b35fc98a87460caeab99fe50fc35fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:25.063645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.103305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.104019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.104303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.104582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] PATH_INFO: `/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d/users/d55b308e21e24490b65cfccc206aec1b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.122332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f4d0abb0254cd6b56ebaddbf975e6d, user_id=d55b308e21e24490b65cfccc206aec1b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.128062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.158209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 756/1545] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:25.158209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.158209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.175116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61038b0-430b-4bbf-9592-89810f53dae5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.181723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.187081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.187081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.187081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] PATH_INFO: `/v3/projects/25de4d4cadff438b82efd4304e9d6fbc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.187081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25de4d4cadff438b82efd4304e9d6fbc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.187081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.249695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 790/1546] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:24 2022] PUT /identity/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d/users/d55b308e21e24490b65cfccc206aec1b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:25.250352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.333150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7ae7e-e104-48cc-9a4c-f1f806c071a9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.338779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.339580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.340014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.341416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.343156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.345513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.366102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25de4d4cadff438b82efd4304e9d6fbc', 'request_id': 'req-778605c0-a1bd-4885-ab6b-46951e5b1994'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.372724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25de4d4cadff438b82efd4304e9d6fbc', 'request_id': 'req-778605c0-a1bd-4885-ab6b-46951e5b1994'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.454367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:25.482610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25de4d4cadff438b82efd4304e9d6fbc', 'request_id': 'req-778605c0-a1bd-4885-ab6b-46951e5b1994'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.490206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 791/1547] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:25 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:25.491385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.495226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] The token cache is being invalidate because project 25de4d4cadff438b82efd4304e9d6fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:25.495520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.496182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.497929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 757/1548] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:25 2022] DELETE /identity/v3/projects/25de4d4cadff438b82efd4304e9d6fbc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:25.498559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.499826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.507510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:25.555551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61314e08-0dd3-469f-bad6-c0b90b8566c4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.560164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.561446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.561782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.562004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.562671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.564340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.565982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778605c0-a1bd-4885-ab6b-46951e5b1994 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.570902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.571540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.571825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.572104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] PATH_INFO: `/v3/users/fb06f0df47934195aad5e10d2489ff12` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.573264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb06f0df47934195aad5e10d2489ff12)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.575741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.576250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 792/1549] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:25.577276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.579072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.592162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673419a-2d6c-4f1c-8bb1-97477864199d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.598325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.598993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.599283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.601178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.601178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.602703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.696946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 793/1550] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:25 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:25.697729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.700565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.713319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93e96f-7d35-4b5c-bc07-f4a33e05e8b9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:25.719166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:25.719812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:25.720093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:25.720365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] PATH_INFO: `/v3/users/90407be7382446b3a8909ba4d32fea3a` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:25.721500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90407be7382446b3a8909ba4d32fea3a)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:25.723623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:25.797866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb06f0df47934195aad5e10d2489ff12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.798556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb06f0df47934195aad5e10d2489ff12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.806197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb06f0df47934195aad5e10d2489ff12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.819980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb06f0df47934195aad5e10d2489ff12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.915628 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb06f0df47934195aad5e10d2489ff12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.922832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 758/1551] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:25 2022] DELETE /identity/v3/users/fb06f0df47934195aad5e10d2489ff12 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:25.925815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:25.927428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:25.942521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90407be7382446b3a8909ba4d32fea3a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.950836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90407be7382446b3a8909ba4d32fea3a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.966890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90407be7382446b3a8909ba4d32fea3a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.967848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90407be7382446b3a8909ba4d32fea3a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:25.997254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862a1a7-da0f-4dfe-93c1-6eb135906402 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.002088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.002792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.002975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:26.003073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.003351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] PATH_INFO: `/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d/users/d55b308e21e24490b65cfccc206aec1b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.005353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f4d0abb0254cd6b56ebaddbf975e6d, user_id=d55b308e21e24490b65cfccc206aec1b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.008086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.019701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 759/1552] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:25 2022] PUT /identity/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d/users/d55b308e21e24490b65cfccc206aec1b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:26.020516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.022046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.060643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0c4ebedb-5a21-4415-a762-156bf8b700d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90407be7382446b3a8909ba4d32fea3a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:26.068940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 794/1553] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:25 2022] DELETE /identity/v3/users/90407be7382446b3a8909ba4d32fea3a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:26.070212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3f35-2eb9-4ef9-a0f6-de0d96af5096 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.070522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3f35-2eb9-4ef9-a0f6-de0d96af5096 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.070816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3f35-2eb9-4ef9-a0f6-de0d96af5096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.092976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-11fb3f35-2eb9-4ef9-a0f6-de0d96af5096 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:26.094189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-11fb3f35-2eb9-4ef9-a0f6-de0d96af5096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.094837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d442a0e-74c4-415c-aba3-e9ffc2022064 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.160857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.161939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.161939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.162393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 795/1554] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:26.164046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.164718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.165096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.165207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.166037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.167765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.182491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 760/1555] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:26.182491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.182491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.199140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] MFA Rules not processed for user `d55b308e21e24490b65cfccc206aec1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:26.200369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.214082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:26.243425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3b648-b06d-4123-bac5-1b33a07a3fc2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.250755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.250755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.250755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.250755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.259079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.260521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.271095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 761/1556] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:26.271846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 796/1557] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:26.287539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.288881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.294964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.302834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:26.304705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6b93-a48a-4523-b4d4-df5ac195553a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.311005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.312153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.312578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.313013 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] PATH_INFO: `/v3/projects/bc1f85e930644722a0a09c4422e5d025/users/761e54684958454386bcaef98934620e/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.335532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1f85e930644722a0a09c4422e5d025, user_id=761e54684958454386bcaef98934620e, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.339100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.373488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78461d6-7ca5-48f0-aa73-1a1487173c56 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.383990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.384633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.384919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.385196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.400932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.402902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.413288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 797/1558] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:26.423231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 762/1559] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:26 2022] PUT /identity/v3/projects/bc1f85e930644722a0a09c4422e5d025/users/761e54684958454386bcaef98934620e/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:26.425412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.441779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63982987-1e50-4c52-928f-1039129a9ced service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.520780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.521849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.522150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.522425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.523164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.524780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.541516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 798/1560] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:26.546269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.566776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1da1-59b4-459a-82c8-d76d82a92811 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.573839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.574578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.574866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.575139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] PATH_INFO: `/v3/projects/bc1f85e930644722a0a09c4422e5d025/users/761e54684958454386bcaef98934620e/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.577955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1f85e930644722a0a09c4422e5d025, user_id=761e54684958454386bcaef98934620e, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.581835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873ea6-6aba-4ff9-ad29-e4e74b848ab3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.606431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 763/1561] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:26 2022] PUT /identity/v3/projects/bc1f85e930644722a0a09c4422e5d025/users/761e54684958454386bcaef98934620e/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:26.607450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.608761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.609065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.609327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.609436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.624528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbfb92-5483-4846-b0a0-ba02478fe337 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.639590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:26.647552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] MFA Rules not processed for user `761e54684958454386bcaef98934620e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:26.649504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.695862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.696514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.696953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.697239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] PATH_INFO: `/v3/users/871e1c0a885f46c0848ea480f792554a` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.698564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871e1c0a885f46c0848ea480f792554a)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.700592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:26.737991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 764/1562] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:26.739116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:26.740520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:26.756138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c920f99-0fc0-49ff-8589-51841f4d5b04 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:26.760810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:26.762383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:26.762383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:26.762383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] PATH_INFO: `/v3/projects/a26d4b4a9b914da68923ee9d7e47366c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:26.763311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26d4b4a9b914da68923ee9d7e47366c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:26.765529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:27.959647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871e1c0a885f46c0848ea480f792554a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:27.967046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871e1c0a885f46c0848ea480f792554a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:27.973453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26d4b4a9b914da68923ee9d7e47366c', 'request_id': 'req-21d39d13-dab4-40c0-95ed-c9028502a2df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:27.981875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871e1c0a885f46c0848ea480f792554a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:27.981875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871e1c0a885f46c0848ea480f792554a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:27.982083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26d4b4a9b914da68923ee9d7e47366c', 'request_id': 'req-21d39d13-dab4-40c0-95ed-c9028502a2df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.083882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26d4b4a9b914da68923ee9d7e47366c', 'request_id': 'req-21d39d13-dab4-40c0-95ed-c9028502a2df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.097166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] The token cache is being invalidate because project a26d4b4a9b914da68923ee9d7e47366c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:28.097391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.098027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.099538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 765/1563] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:26 2022] DELETE /identity/v3/projects/a26d4b4a9b914da68923ee9d7e47366c => generated 0 bytes in 1361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:28.100374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.103603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.160284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39d13-dab4-40c0-95ed-c9028502a2df admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.164193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.164814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.165096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.165518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.167468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.203323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871e1c0a885f46c0848ea480f792554a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.211890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 799/1564] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:26 2022] DELETE /identity/v3/users/871e1c0a885f46c0848ea480f792554a => generated 0 bytes in 1606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:28.212841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.219394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.259640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.261847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb254c-fd6a-442b-9f33-36d038ab5daa service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.274732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 766/1565] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:28.287776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd70c1-6f82-4fe3-a8a1-b889ed66de41 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.288975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd70c1-6f82-4fe3-a8a1-b889ed66de41 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.290642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd70c1-6f82-4fe3-a8a1-b889ed66de41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.322100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-91dd70c1-6f82-4fe3-a8a1-b889ed66de41 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:28.322100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-91dd70c1-6f82-4fe3-a8a1-b889ed66de41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.326241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 767/1566] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:28.326866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661cb-85b1-424e-8b21-203ce2f35608 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.327074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661cb-85b1-424e-8b21-203ce2f35608 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.327352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424661cb-85b1-424e-8b21-203ce2f35608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.335189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebcd16-8588-4b46-9036-fa86d66d1e46 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.341232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:28.348661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.349313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.349707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.349908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] PATH_INFO: `/v3/projects/434398e05fd246f5b6cd87e874a9b7ac` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.351329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] RBAC: Authorizing `identity:delete_project(project_id=434398e05fd246f5b6cd87e874a9b7ac)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.352004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-424661cb-85b1-424e-8b21-203ce2f35608 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:28.353229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-424661cb-85b1-424e-8b21-203ce2f35608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.353428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.362317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 768/1567] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:28.363596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.363979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.364203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.395099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:28.395099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.404181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 769/1568] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:28.405234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.406660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.464471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f9c1d-87ca-4908-946b-df4b2e14c3d9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.468919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.469639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.469919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.470194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.479358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.480846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.488143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434398e05fd246f5b6cd87e874a9b7ac', 'request_id': 'req-8f78dce6-19fd-46c5-b835-00765c4d981b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.490306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 770/1569] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:28.491866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.503593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a1144-a1a7-46e5-9025-d6b5192ced26 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.511374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434398e05fd246f5b6cd87e874a9b7ac', 'request_id': 'req-8f78dce6-19fd-46c5-b835-00765c4d981b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.573593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.574370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.575377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.576092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.576381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:28.576690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.578286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.591295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 771/1570] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:28.595640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.601214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434398e05fd246f5b6cd87e874a9b7ac', 'request_id': 'req-8f78dce6-19fd-46c5-b835-00765c4d981b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.605686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ab402-8f5c-47ca-b48d-44ad01f255da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.610615 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.611216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.611504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.611778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.612406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.613090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] The token cache is being invalidate because project 434398e05fd246f5b6cd87e874a9b7ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:28.613384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.614019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.614311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.615798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 800/1571] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:28 2022] DELETE /identity/v3/projects/434398e05fd246f5b6cd87e874a9b7ac => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:28.619757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.621071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.634193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78dce6-19fd-46c5-b835-00765c4d981b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.639507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.640131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.640413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.640688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] PATH_INFO: `/v3/users/04f4910fb1b94b99ab10dd1670a0e59d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.641938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f4910fb1b94b99ab10dd1670a0e59d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.643894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.707705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 772/1572] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:28 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:28.713840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.713840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.801390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6469a5-4cdf-4627-9fc9-0a0041025c0a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.806868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.811593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.811593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.811593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.811593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.812049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.927129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 773/1573] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:28 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:28.928324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.939999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4260-4fec-42b9-af36-39406691bc98 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.940736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f4910fb1b94b99ab10dd1670a0e59d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.946040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.946753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.947269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.947589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.948301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.948992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f4910fb1b94b99ab10dd1670a0e59d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.950671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:28.961038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 774/1574] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:28.963555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f4910fb1b94b99ab10dd1670a0e59d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.964493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f4910fb1b94b99ab10dd1670a0e59d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:28.964930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:28.966639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:28.979504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164b70b-c7df-4bd6-b8a3-4a42f2f2ff09 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:28.984399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:28.985015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:28.985325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:28.985647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:28.986462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:28.989671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.000842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 775/1575] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:29.003581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.006252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.008994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:29.035690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed288286-243a-40df-a001-0e9e3c5d2e17 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:29.040552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:29.041192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.041475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.041751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] PATH_INFO: `/v3/projects/5c8ddc1d11ef4c01a5a134b2612bc363` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.042917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c8ddc1d11ef4c01a5a134b2612bc363)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:29.044859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.076932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f4910fb1b94b99ab10dd1670a0e59d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.083452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 801/1576] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:28 2022] DELETE /identity/v3/users/04f4910fb1b94b99ab10dd1670a0e59d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:29.084294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.148869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc435dd5-3819-4386-b6f2-b108f84f2612 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:29.154695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:29.155736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.156028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.156306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] PATH_INFO: `/v3/projects/9feb901e5a114f008dc3d94046ffa99d/users/d41ccac0df8e4decb4496a000ee78d1c/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.171001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9feb901e5a114f008dc3d94046ffa99d, user_id=d41ccac0df8e4decb4496a000ee78d1c, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:29.173848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eac23-76f3-4c24-b714-d6226611bfc1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.266916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 802/1577] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:29 2022] PUT /identity/v3/projects/9feb901e5a114f008dc3d94046ffa99d/users/d41ccac0df8e4decb4496a000ee78d1c/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:29.275032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c8ddc1d11ef4c01a5a134b2612bc363', 'request_id': 'req-724caf37-8feb-476b-8390-a823988bbb61'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.277118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.277478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.278484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.281727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c8ddc1d11ef4c01a5a134b2612bc363', 'request_id': 'req-724caf37-8feb-476b-8390-a823988bbb61'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.314159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] MFA Rules not processed for user `d41ccac0df8e4decb4496a000ee78d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:29.316413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.401896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 803/1578] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:29.401896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.401896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.410194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:29.411139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c8ddc1d11ef4c01a5a134b2612bc363', 'request_id': 'req-724caf37-8feb-476b-8390-a823988bbb61'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.430723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] The token cache is being invalidate because project 5c8ddc1d11ef4c01a5a134b2612bc363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:29.431021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.432584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.435590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 776/1579] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:29 2022] DELETE /identity/v3/projects/5c8ddc1d11ef4c01a5a134b2612bc363 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:29.436795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.448174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.515458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a13d2-a483-49eb-a0a0-7719896bddf5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:29.528373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:29.529942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.530770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.531588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] PATH_INFO: `/v3/users/dc358c517c7346caba17ec870678ff16` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.533645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc358c517c7346caba17ec870678ff16)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:29.536836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.556574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724caf37-8feb-476b-8390-a823988bbb61 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:29.562459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:29.564009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.564523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.565553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.566323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:29.568650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.695586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 777/1580] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:29 2022] POST /identity/v3/projects => generated 375 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:29.696322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.698018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.765209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390e1af-ab47-4a28-93c5-7d9b2cd6d003 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:29.770146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:29.770815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:29.771100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:29.771374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] PATH_INFO: `/v3/users/d55b308e21e24490b65cfccc206aec1b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:29.772565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55b308e21e24490b65cfccc206aec1b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:29.774480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:29.824356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc358c517c7346caba17ec870678ff16'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.831631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc358c517c7346caba17ec870678ff16'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.845388 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc358c517c7346caba17ec870678ff16'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.846152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc358c517c7346caba17ec870678ff16'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.933973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc358c517c7346caba17ec870678ff16'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.944732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 804/1581] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:29 2022] DELETE /identity/v3/users/dc358c517c7346caba17ec870678ff16 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:29.944732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:29.944732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:29.994970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55b308e21e24490b65cfccc206aec1b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:29.997085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55b308e21e24490b65cfccc206aec1b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.027230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55b308e21e24490b65cfccc206aec1b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.062855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55b308e21e24490b65cfccc206aec1b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.096863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa922-e5c2-42a2-b569-d46552c1c3e1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.112645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '761e54684958454386bcaef98934620e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc1f85e930644722a0a09c4422e5d025', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.113668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.114021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.116461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.120038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:30.120373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:30.123733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:30.152729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 805/1582] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:30.153622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:30.155834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.167573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55b308e21e24490b65cfccc206aec1b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.184641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 778/1583] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:29 2022] DELETE /identity/v3/users/d55b308e21e24490b65cfccc206aec1b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:30.186338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:30.188662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.324103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababd875-b2fa-4888-9aa0-e0f44931ab58 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.329988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.330687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.330792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eaf2f-d3dd-4270-828a-8240665b92ba admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.330978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.331418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] PATH_INFO: `/v3/projects/d555cc0b47f841198e3a21867df20a64` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.332642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d555cc0b47f841198e3a21867df20a64)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:30.335155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:30.349769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.354690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.355535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.356386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.359675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:30.359675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc428fe-4aef-4e60-838d-bdfe4f052f9b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:30.511037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 779/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:30 2022] POST /identity/v3/users => generated 294 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:30.512877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.517715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.517715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.533585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:30.534848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.547464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 780/1585] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:30.557867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:30.559369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.634975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc356c4-8572-4ac5-a0e1-e500ac4ec4aa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.646762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd555cc0b47f841198e3a21867df20a64', 'request_id': 'req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.655117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd555cc0b47f841198e3a21867df20a64', 'request_id': 'req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.707969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '761e54684958454386bcaef98934620e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc1f85e930644722a0a09c4422e5d025', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.707969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.707969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.708554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.708554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:30.708554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:30.763040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd555cc0b47f841198e3a21867df20a64', 'request_id': 'req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.777615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] The token cache is being invalidate because project d555cc0b47f841198e3a21867df20a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:30.778028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.778805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e86dc5a8-7829-4928-bb9a-b5cae0ba5632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:30.780476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 806/1586] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:30 2022] DELETE /identity/v3/projects/d555cc0b47f841198e3a21867df20a64 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:30.781771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.782269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.782931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.784317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 807/1587] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:35:30.785110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:30.787014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.838926 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 781/1588] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:30 2022] POST /identity/v3/projects => generated 354 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:30.841198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:30.843219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.885642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4884bb4-0310-4300-826f-4eb695cdf0f0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.917490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599926bf-bee5-4e28-b81e-54cb61ad8573 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:30.922108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.923316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.923747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.924019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.925903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.955581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:30.956153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.956561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.956843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.957691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:30.959401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc6c33-9441-4ab1-948c-05ed58f78e3b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:30.972467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 808/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:30.973672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec026-b99e-4e41-884c-2ca03dec747a None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:30.974260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec026-b99e-4e41-884c-2ca03dec747a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:30.974730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ec026-b99e-4e41-884c-2ca03dec747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:30.996198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-313ec026-b99e-4e41-884c-2ca03dec747a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:30.997968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-313ec026-b99e-4e41-884c-2ca03dec747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:30.999443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.000889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.007274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 809/1590] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:31.008545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.008815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.009085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.012680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 782/1591] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:31.014774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.016403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.036926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:31.038130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.088631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08f9e7-11c1-4179-8bb5-e44b61f4c815 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.093388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '761e54684958454386bcaef98934620e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc1f85e930644722a0a09c4422e5d025', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.094019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.094300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.094896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.095255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:31.095556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.097147 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.111438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 783/1592] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:31.112231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.113908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.129676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8592c-ff6a-4a5a-b40d-0c2c93d341fa tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.129929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 810/1593] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:31.130985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.138985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.138985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.138985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.138985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] PATH_INFO: `/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.139320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f4d0abb0254cd6b56ebaddbf975e6d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.140406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.143059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468477-1454-474e-a4f0-56ee0998b166 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.147983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.148646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.148926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.149198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] PATH_INFO: `/v3/projects/eb52c0c496134b448b8023161f2d804b/users/e0858fbb3c6e47829e64e0fb96029977/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.169523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb52c0c496134b448b8023161f2d804b, user_id=e0858fbb3c6e47829e64e0fb96029977, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.172273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6b9b7-0158-4461-9dc5-b36f870ce7f7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.259553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 811/1594] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:31 2022] PUT /identity/v3/projects/eb52c0c496134b448b8023161f2d804b/users/e0858fbb3c6e47829e64e0fb96029977/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:31.259553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.259553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.259553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.283398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:31.288381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.288500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f4d0abb0254cd6b56ebaddbf975e6d', 'request_id': 'req-8ad6e13d-073e-45f0-be1e-a2441ae54775'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.300928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f4d0abb0254cd6b56ebaddbf975e6d', 'request_id': 'req-8ad6e13d-073e-45f0-be1e-a2441ae54775'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.325509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:31.364617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 812/1595] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:31.365807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.367570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.384289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f4d0abb0254cd6b56ebaddbf975e6d', 'request_id': 'req-8ad6e13d-073e-45f0-be1e-a2441ae54775'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.397571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] The token cache is being invalidate because project 44f4d0abb0254cd6b56ebaddbf975e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:31.397957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.398745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.400346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 784/1596] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:31 2022] DELETE /identity/v3/projects/44f4d0abb0254cd6b56ebaddbf975e6d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:31.401355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.402975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.414783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:31.446235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44e738-3efe-495b-b669-8e9fd9355ca9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.451768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.452374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.452674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.452925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.454682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.467166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.468757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.480456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 813/1597] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:31.481834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.483284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.497454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392b8e-e6b0-4cb8-8edc-e920ee0fe3d3 service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.500135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6e13d-073e-45f0-be1e-a2441ae54775 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.502452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.503180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.503458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.503735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.504365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.505183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '761e54684958454386bcaef98934620e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc1f85e930644722a0a09c4422e5d025', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.505827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.505991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.506083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.506344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] PATH_INFO: `/v3/projects/2898910dc7e44cd689673e8afa65c520` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.507504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2898910dc7e44cd689673e8afa65c520)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.509385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.517016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 814/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:31.517802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.521765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.535961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e84c2-c7d7-48fc-8590-97e3fc18dabd admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.541678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.542861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.543171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.543669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.544694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:31.545037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.546652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eba453-d43d-4271-8229-0cd7e8d49a76 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.559389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 815/1599] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:31.560558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.560837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.561361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.580217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:31.581422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.589272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 816/1600] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:31.590071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.591839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.602905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f20a00-8e19-4299-b56b-b48b499ef7e4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.607460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.608051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.608333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.608608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] PATH_INFO: `/v3/projects/c3aadaa99dd54ed59d4d752d0804e39d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.609707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3aadaa99dd54ed59d4d752d0804e39d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.611618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.655499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2898910dc7e44cd689673e8afa65c520', 'request_id': 'req-5d73a827-0971-4934-aa8d-c2c68db06704'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.660973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2898910dc7e44cd689673e8afa65c520', 'request_id': 'req-5d73a827-0971-4934-aa8d-c2c68db06704'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.699035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3aadaa99dd54ed59d4d752d0804e39d', 'request_id': 'req-324bab9d-ca2b-4945-8f1b-2bde8e812db2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.704830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3aadaa99dd54ed59d4d752d0804e39d', 'request_id': 'req-324bab9d-ca2b-4945-8f1b-2bde8e812db2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.722618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2898910dc7e44cd689673e8afa65c520', 'request_id': 'req-5d73a827-0971-4934-aa8d-c2c68db06704'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.734554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] The token cache is being invalidate because project 2898910dc7e44cd689673e8afa65c520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:31.734779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.735431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.736964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 785/1601] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:31 2022] DELETE /identity/v3/projects/2898910dc7e44cd689673e8afa65c520 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:31.737744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.739068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.790974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3aadaa99dd54ed59d4d752d0804e39d', 'request_id': 'req-324bab9d-ca2b-4945-8f1b-2bde8e812db2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.803135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] The token cache is being invalidate because project c3aadaa99dd54ed59d4d752d0804e39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:31.803429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.804126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:31.805404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a827-0971-4934-aa8d-c2c68db06704 tempest-VolumeQuotaClassesTest-1878071244 tempest-VolumeQuotaClassesTest-1878071244-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.809332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 817/1602] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:31 2022] DELETE /identity/v3/projects/c3aadaa99dd54ed59d4d752d0804e39d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:31.810183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:31.813003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:31.874813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.875662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.876070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.876759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] PATH_INFO: `/v3/projects/eb52c0c496134b448b8023161f2d804b/users/e0858fbb3c6e47829e64e0fb96029977/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.879381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb52c0c496134b448b8023161f2d804b, user_id=e0858fbb3c6e47829e64e0fb96029977, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.882779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870270d-c6d5-4d1e-867a-ce6245bf386a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.892510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bab9d-ca2b-4945-8f1b-2bde8e812db2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:31.898339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:31.899128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.899577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.899863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.900508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:31.902216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 786/1603] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:31 2022] PUT /identity/v3/projects/eb52c0c496134b448b8023161f2d804b/users/e0858fbb3c6e47829e64e0fb96029977/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:31.904156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc30f-a5c4-470e-bce9-da05b145da07 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:31.904591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc30f-a5c4-470e-bce9-da05b145da07 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:31.905059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bc30f-a5c4-470e-bce9-da05b145da07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:31.911784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:31.938383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc6bc30f-a5c4-470e-bce9-da05b145da07 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:31.944874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc6bc30f-a5c4-470e-bce9-da05b145da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.031381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 818/1604] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:31 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:32.033120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.037104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.055263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 787/1605] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:32.056797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.057097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.057498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.089093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] MFA Rules not processed for user `e0858fbb3c6e47829e64e0fb96029977`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:32.090340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.124193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142b509-9dcc-4bbf-abc2-c8d6fb4f74ad admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.129365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.130120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.130681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.130790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] PATH_INFO: `/v3/users/444afc6c39fe4d35acfe2b227f64e675` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.132223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444afc6c39fe4d35acfe2b227f64e675)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.134239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.172109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 788/1606] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:32.172109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.183327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d368b-21c6-44d0-8bfd-f7f9a898cefb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.189293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.189560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.189835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.190104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.190935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.192835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.335255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 789/1607] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:32 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:32.336182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.345652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e1776-2d04-4f8b-9c4e-9b71b1ab751b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.350312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.350924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.351207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.351482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.352120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:32.352424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.354124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.363650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:32.368374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 790/1608] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:32.369106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.373571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.379379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444afc6c39fe4d35acfe2b227f64e675'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.386879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444afc6c39fe4d35acfe2b227f64e675'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.400008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444afc6c39fe4d35acfe2b227f64e675'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.400746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444afc6c39fe4d35acfe2b227f64e675'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.434791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956aa943-0bfb-46ad-8a7d-6fec7f73dfd1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.438954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.439568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.439849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.440113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.441872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.482231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444afc6c39fe4d35acfe2b227f64e675'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.490071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 819/1609] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:32 2022] DELETE /identity/v3/users/444afc6c39fe4d35acfe2b227f64e675 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:32.494916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.494916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.514827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.516361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.540318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 791/1610] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:32.541604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.544424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.578572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbc7e-f569-497d-9159-05080d139214 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.597701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:32.618300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae8569-a076-40c1-9bfc-19018f446051 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.623884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.624696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.624984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.625263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.625975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.627970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.656046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 820/1611] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:32.656965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.658879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:32.672449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a3f06-a359-4fec-9374-5c6b3b5064ec admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.677708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.678341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.678664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.678941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] PATH_INFO: `/v3/users/33b35fc98a87460caeab99fe50fc35fb` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.680287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b35fc98a87460caeab99fe50fc35fb)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.681226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.681839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.681994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.682256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.682356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.682958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.684962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.823627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 792/1612] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:32 2022] POST /identity/v3/projects => generated 367 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:32.825754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.957737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710efd2-3a8c-4de4-bc27-1b0a1eaa9277 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:32.959296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b35fc98a87460caeab99fe50fc35fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.963498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:32.964220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:32.964532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:32.964804 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:32.965620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:32.967633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:32.968391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b35fc98a87460caeab99fe50fc35fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.982364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 793/1613] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:32.982364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:32.986371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b35fc98a87460caeab99fe50fc35fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.988005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b35fc98a87460caeab99fe50fc35fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:32.991486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b879831-d5ff-4738-a402-ebf156b706fb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.005980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.006770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.007090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.007501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.008228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.011326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.082872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b35fc98a87460caeab99fe50fc35fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.094046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 821/1614] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:32 2022] DELETE /identity/v3/users/33b35fc98a87460caeab99fe50fc35fb => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:33.094046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.096934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.125317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 794/1615] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:32 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:33.127345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.133448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.163498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eac0aa-4443-468e-b69e-b9262dcead19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.227546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53623f0-016a-47ce-b4d5-2e1fa95553ca admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.234199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.246824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.246824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.246824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] PATH_INFO: `/v3/projects/7231e0fd898e4ad39fa9f7de5a66ea5e/users/e81c0391ae3445828de1b83b5ce1bd0e/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.252487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7231e0fd898e4ad39fa9f7de5a66ea5e, user_id=e81c0391ae3445828de1b83b5ce1bd0e, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.255561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095761b-2837-4155-8701-19151ec7831a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.270552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.271203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.271484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.271757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] PATH_INFO: `/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.272916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61eb9c1e6c6e4174bc9b176a31320bf3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.275205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.381320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 822/1616] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:33 2022] PUT /identity/v3/projects/7231e0fd898e4ad39fa9f7de5a66ea5e/users/e81c0391ae3445828de1b83b5ce1bd0e/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:33.383203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.383496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.383773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.408523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:33.410720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.431048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61eb9c1e6c6e4174bc9b176a31320bf3', 'request_id': 'req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.435122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61eb9c1e6c6e4174bc9b176a31320bf3', 'request_id': 'req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.489167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 823/1617] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:33.493280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.494637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.506664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3073-0cfb-4fff-9558-1ad584364963 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.516116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.519305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.519305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.519305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.519305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.519702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd302b2-20f7-4c02-8c95-196827c46949 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.529917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 824/1618] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:33.534154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.534708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.534708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.539133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61eb9c1e6c6e4174bc9b176a31320bf3', 'request_id': 'req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.559779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] The token cache is being invalidate because project 61eb9c1e6c6e4174bc9b176a31320bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:33.564087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.564902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:33.570876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 795/1619] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:33 2022] DELETE /identity/v3/projects/61eb9c1e6c6e4174bc9b176a31320bf3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:33.575713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.575841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:33.577716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.578600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.591286 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 825/1620] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:33.591994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.593368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.610598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a90ea-72e6-4676-a06f-e715f5e17353 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.614706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.615352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.615633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.615908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.616557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.618276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a85e5-ff7b-40c4-a740-52274a7030f0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.629112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 826/1621] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:33.630338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.630647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.630923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.649845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:33.652053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.657113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73cb51-bd1c-46c9-91cf-1e671b4f93c2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.662594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.663339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.663744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.664197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.666459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.667284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 827/1622] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:33.668742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.683294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aaed8-1ef4-4681-aa8a-2ec3821b7213 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.683584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.685953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.689819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.690443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.690779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.691207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.696670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 796/1623] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:33.697398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.706884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c0be0-d637-4978-a4e7-8ac03ac1404f service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.713019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.713717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.713853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795984ba-b00d-4725-b091-7e54b3f965c9 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.714841 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.715403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.715757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] PATH_INFO: `/v3/projects/5cdb28c69bb84c5a8bee83efb2feaf9e/users/2627eb839af34a3f9e6b4b7234caf44f/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.726568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 828/1624] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:33.727936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.728221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.729599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.737639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdb28c69bb84c5a8bee83efb2feaf9e, user_id=2627eb839af34a3f9e6b4b7234caf44f, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.741931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673852c-a9e0-4b20-a0f2-e4a5eb27840b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.752613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:33.753851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.762031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 829/1625] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:33.766294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.776174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c28e9f-6841-4ab1-bd95-6ad0c24344c0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.781268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.781948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.782229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.782496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.783331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:33.783695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.785296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.801479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 830/1626] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:33.806122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.825173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b608e44-fabc-4c8a-bdd8-5e947e9138cb admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.833001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 797/1627] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:33 2022] PUT /identity/v3/projects/5cdb28c69bb84c5a8bee83efb2feaf9e/users/2627eb839af34a3f9e6b4b7234caf44f/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:33.856153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.856153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.856418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.886761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] MFA Rules not processed for user `2627eb839af34a3f9e6b4b7234caf44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:33.888148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:33.890158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.890920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.891312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.891692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.892431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.894537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:33.960196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 798/1628] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:33.964993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:33.976137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e660f-4c06-41bc-a389-021db41d1550 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:33.981938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:33.982746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:33.983033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:33.983304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:33.983945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:33.985861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.008614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 831/1629] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:34.018711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.090434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 799/1630] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:33 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:34.094376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.107857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10af98-8012-43e7-b4a1-84792f615eac admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.198116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.198778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.199080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.199625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.200022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.202116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.245143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f013fe-69f5-4382-a002-f381f681f82c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.250525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.251152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.251437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.251715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.252889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.254364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.375487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 832/1631] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:34 2022] POST /identity/v3/users => generated 293 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:34.380270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.382462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:34.395930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2117a-f124-4e37-8e92-8dd5c4c9dd69 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.400632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.412677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.413039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.415649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] PATH_INFO: `/v3/projects/b0e5776637bc4aceb3bccdca0f7e480f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.418543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 800/1632] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:34 2022] POST /identity/v3/users => generated 290 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:34.419016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.420937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e5776637bc4aceb3bccdca0f7e480f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.422916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.441290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d1de-c1a3-489d-be90-f7d5fb04e1dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.449817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.450113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.450397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.450741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.451379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.455153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.467474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 801/1633] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:34.468430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.492412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29cafa-03e0-4582-be62-3757df7a7323 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.497439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.498103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.498586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.498869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.499515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.501226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.514218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 802/1634] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:34.515200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.526230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65640310-8158-4f39-a586-a6008632a224 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.533180 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.534338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.534739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.535089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.535838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.538156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.549437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 803/1635] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:34.550441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.563511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ce32-cb90-4367-81da-c61ea2b5c124 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.569074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.569834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.570125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.570406 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] PATH_INFO: `/v3/projects/25e12b2be27d46e3ac1ea01a75a52941/users/9949ff8d2ab14ebd8a17b9560ced00e7/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.600668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e12b2be27d46e3ac1ea01a75a52941, user_id=9949ff8d2ab14ebd8a17b9560ced00e7, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.603409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.769087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 804/1636] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:34 2022] PUT /identity/v3/projects/25e12b2be27d46e3ac1ea01a75a52941/users/9949ff8d2ab14ebd8a17b9560ced00e7/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:34.770255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.778647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e5776637bc4aceb3bccdca0f7e480f', 'request_id': 'req-4445f548-b100-434c-b5d8-9b6c5a7c77cf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:34.784597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e5776637bc4aceb3bccdca0f7e480f', 'request_id': 'req-4445f548-b100-434c-b5d8-9b6c5a7c77cf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:34.820196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:34.835366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2776cc8-718f-4a9d-9135-9ecf4700fa4b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.861308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e5776637bc4aceb3bccdca0f7e480f', 'request_id': 'req-4445f548-b100-434c-b5d8-9b6c5a7c77cf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:34.885595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] The token cache is being invalidate because project b0e5776637bc4aceb3bccdca0f7e480f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:34.885889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:34.888257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:34.890501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 833/1637] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:34 2022] DELETE /identity/v3/projects/b0e5776637bc4aceb3bccdca0f7e480f => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:34.891308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.894260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:34.917668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.919655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.920277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.921300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] PATH_INFO: `/v3/projects/a1540757eff14b6fb6b9cbedb29da293/users/24fc111e6b0447b39eaabb2c387ed7b1/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.949209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1540757eff14b6fb6b9cbedb29da293, user_id=24fc111e6b0447b39eaabb2c387ed7b1, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.952508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.971380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f548-b100-434c-b5d8-9b6c5a7c77cf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:34.978128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:34.978591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:34.978969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:34.979345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:34.981583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:34.984809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:34.993435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 834/1638] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:34.994276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:34.996448 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.016373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80354e2d-2b7f-4589-b2fa-b73cf518e2aa admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.024014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.024733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.025160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.025584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] PATH_INFO: `/v3/users/761e54684958454386bcaef98934620e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.026789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=761e54684958454386bcaef98934620e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.030134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.033967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:35.044454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 805/1639] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:34 2022] PUT /identity/v3/projects/a1540757eff14b6fb6b9cbedb29da293/users/24fc111e6b0447b39eaabb2c387ed7b1/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:35.047110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.131432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef2418-8a84-4f4d-800f-310241b0af11 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.137271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.138054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.146527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.146527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] PATH_INFO: `/v3/projects/25e12b2be27d46e3ac1ea01a75a52941/users/9949ff8d2ab14ebd8a17b9560ced00e7/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.146527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e12b2be27d46e3ac1ea01a75a52941, user_id=9949ff8d2ab14ebd8a17b9560ced00e7, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.146527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50882892-2802-4f27-a983-74de4f3db829 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.155407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 806/1640] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:35 2022] PUT /identity/v3/projects/25e12b2be27d46e3ac1ea01a75a52941/users/9949ff8d2ab14ebd8a17b9560ced00e7/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:35.156655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb2672-0565-4a7d-b020-445c91fc8ad7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.156958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb2672-0565-4a7d-b020-445c91fc8ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.157220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb2672-0565-4a7d-b020-445c91fc8ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.184775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-01cb2672-0565-4a7d-b020-445c91fc8ad7 None None] MFA Rules not processed for user `24fc111e6b0447b39eaabb2c387ed7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:35.186225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-01cb2672-0565-4a7d-b020-445c91fc8ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.250865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 807/1641] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:35.252165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.252451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.252724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.279923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761e54684958454386bcaef98934620e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:35.280731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] MFA Rules not processed for user `9949ff8d2ab14ebd8a17b9560ced00e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:35.281952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.289433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761e54684958454386bcaef98934620e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:35.317494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761e54684958454386bcaef98934620e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:35.318247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761e54684958454386bcaef98934620e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:35.353111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 808/1642] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:35.354096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.355601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.408633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761e54684958454386bcaef98934620e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:35.417231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 835/1643] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:34 2022] DELETE /identity/v3/users/761e54684958454386bcaef98934620e => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:35.418627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.428031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.431964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d39c-5d6b-4432-8c0a-ce55cf9efb18 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.513631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.514294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.514833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.515241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.515982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.518554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5650c82-b40b-4910-9479-c79c82680e8d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.536679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898a5e0-9b65-474d-b8d2-18a3584ef2b2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.541244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.542037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.542321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.542664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.604930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.608330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.623174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 809/1644] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:35.624547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.624834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.625113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.625480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 836/1645] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:35.626635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.641603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a2d7-b20c-4ce9-b6e1-4e43d3c706ad service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.681414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] MFA Rules not processed for user `e81c0391ae3445828de1b83b5ce1bd0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:35.682814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.782100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.788690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.788690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.788690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.788690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.788690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.820318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 810/1646] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:35.825068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.826408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:35.841091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0736b-f133-4f85-8797-aeec5a2a10f5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.846347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.847094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.847326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.847529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.848166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.850549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.894045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 837/1647] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/users => generated 299 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:35.897710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.910382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdcfe3-5749-4846-953d-83f6fbab563c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:35.915545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:35.916634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:35.916634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:35.917147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:35.918015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:35.919899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:35.939108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 811/1648] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:35.940201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.953359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 838/1649] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:35.953519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:35.969877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a356be-b93c-46e7-9bcf-cd153752e107 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.013933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929902c0-1e0b-41a0-92dd-032807fa8007 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.020495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.021130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.021419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.021694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.023579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.079163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.080740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.081086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.081362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.082036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.084026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.105616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.114761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.126353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 812/1650] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:36.135693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.159460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6177136-7135-417d-ae97-fe27bb22fd8f service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.162230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.163119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.164103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.164481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] PATH_INFO: `/v3/projects/19768d3c9d044003a54961db4bbff213/users/a0db236508fd44eab626c07a659bc576/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.181165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19768d3c9d044003a54961db4bbff213, user_id=a0db236508fd44eab626c07a659bc576, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.184285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.217772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 839/1651] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:35 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:36.219094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.223752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.264339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098b74-93f3-4a3b-a09b-c04073c3fd8d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.269607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.270707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.271130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.271702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] PATH_INFO: `/v3/projects/bc1f85e930644722a0a09c4422e5d025` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.277643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1f85e930644722a0a09c4422e5d025)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.279652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.291053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 813/1652] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:36 2022] PUT /identity/v3/projects/19768d3c9d044003a54961db4bbff213/users/a0db236508fd44eab626c07a659bc576/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:36.291973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.366762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626368aa-9c93-460d-88e5-7b18a8ab5011 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.372007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.372617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.372902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.373178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.373824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.375499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.385474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 814/1653] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:36.386414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.399265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737de32c-f115-4649-8181-242bfeefbbbc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.403913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.404519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.404802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.405078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.405714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.407358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.420979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 815/1654] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:36.422095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.433503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad8c8-8a57-4960-8c63-3ec26d396a57 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.441192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1f85e930644722a0a09c4422e5d025', 'request_id': 'req-ca672a53-4279-46a5-9991-30f48f796efe'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:36.447044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1f85e930644722a0a09c4422e5d025', 'request_id': 'req-ca672a53-4279-46a5-9991-30f48f796efe'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:36.471457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:36.493577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.494293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.494557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.494832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] PATH_INFO: `/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf/users/04d43b47c7064ba09f0f3d98eac0574b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.511486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=00056f9bb4634b4d880254f0f9e6bfaf, user_id=04d43b47c7064ba09f0f3d98eac0574b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.514723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b610bf-6924-4f5c-b5fb-0a89582c3eec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.516791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1f85e930644722a0a09c4422e5d025', 'request_id': 'req-ca672a53-4279-46a5-9991-30f48f796efe'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:36.529449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] The token cache is being invalidate because project bc1f85e930644722a0a09c4422e5d025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:36.529743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:36.530415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:36.531961 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 840/1655] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:36 2022] DELETE /identity/v3/projects/bc1f85e930644722a0a09c4422e5d025 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:36.532708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.536180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.548772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca672a53-4279-46a5-9991-30f48f796efe admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.553426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.554099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.554375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.554707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] PATH_INFO: `/v3/projects/19768d3c9d044003a54961db4bbff213/users/a0db236508fd44eab626c07a659bc576/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.556559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19768d3c9d044003a54961db4bbff213, user_id=a0db236508fd44eab626c07a659bc576, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.559741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cccd3e-8ff3-4c2e-8e87-95a7bef8c734 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.570027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 841/1656] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:36 2022] PUT /identity/v3/projects/19768d3c9d044003a54961db4bbff213/users/a0db236508fd44eab626c07a659bc576/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:36.571178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.571462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.571737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.591673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:36.592859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.626247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:36.633744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 816/1657] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:36 2022] PUT /identity/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf/users/04d43b47c7064ba09f0f3d98eac0574b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:36.635936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.635936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.635936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.662922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 842/1658] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:36.663827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.665222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.677624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d8bc-7ef6-41ba-8684-1a1a28b6054b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.680272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] MFA Rules not processed for user `a0db236508fd44eab626c07a659bc576`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:36.683482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.739030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.739640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.739920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.740193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.740830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.742598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598698-a040-4f11-9845-3e7a2e15e783 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.756594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 843/1659] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:36.758341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.758720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.759033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.760958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 817/1660] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:36.762271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.781117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03315c7c-b501-4da9-9cc8-acc133af3a7e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.787377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.788244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.788755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.789213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] PATH_INFO: `/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf/users/04d43b47c7064ba09f0f3d98eac0574b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.791385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00056f9bb4634b4d880254f0f9e6bfaf, user_id=04d43b47c7064ba09f0f3d98eac0574b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.795044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.800504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:36.801716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.811580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 818/1661] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:36 2022] PUT /identity/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf/users/04d43b47c7064ba09f0f3d98eac0574b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:36.813084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.817069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:36.885051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 844/1662] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:36.889150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.890249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e93ed-e985-4a94-8b85-f97251c17d67 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.894340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.895652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.896044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.896423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.913445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888feca3-ae10-47bb-aeba-b5334ee4cf70 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.919932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.920654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.920871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.921148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.921811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:36.922277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.923989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.941017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 845/1663] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:36.945512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:36.955768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2afb-841d-45b2-8bfd-e677a232c935 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:36.962924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:36.963230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:36.963568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:36.963854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:36.964554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.966470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:36.997499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:36.999076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.034410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 819/1664] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:37.131227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.136221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 846/1665] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:36 2022] POST /identity/v3/projects => generated 393 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:37.145772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:37.159608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507362-b88e-4f78-b2c7-6527d6253db8 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:37.231818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:37.232704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.232991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.233283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.233993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:37.236005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.392575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 820/1666] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:37 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:37.394385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.408644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8e711-b059-41fb-8d02-609334bf2df7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:37.421510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:37.421510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.421510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.421510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.421841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:37.421841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.428691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 847/1667] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:37.432681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.445012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c2d5f-0c45-4fbd-bd8f-01f76aa5436c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:37.451829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:37.452512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.453268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.453623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.454394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:37.456112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375d36-f533-4edb-a83c-dbfce4d4aa41 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.470490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 821/1668] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:37.471765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.482102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada87a7a-a15f-4102-bef3-48382fc3892b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:37.486842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:37.487680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.487964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.488401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] PATH_INFO: `/v3/projects/fc7f3a2502234199bb98921d65a826ff/users/7423f26ebc1f451ca26bdffa7b67585c/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.506767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7f3a2502234199bb98921d65a826ff, user_id=7423f26ebc1f451ca26bdffa7b67585c, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:37.509479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.587805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 848/1669] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:37 2022] PUT /identity/v3/projects/fc7f3a2502234199bb98921d65a826ff/users/7423f26ebc1f451ca26bdffa7b67585c/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:37.599950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.600246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.600560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.628158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] MFA Rules not processed for user `7423f26ebc1f451ca26bdffa7b67585c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:37.629376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:37.712820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 822/1670] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:37.717768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.797977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88336d-3e92-4cd3-a41e-efcf6e5fe55b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:37.802727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:37.803346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:37.803626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:37.803898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:37.804524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:37.806614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:37.963333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:37.977331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:37.982577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 849/1671] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:37 2022] POST /identity/v3/projects => generated 393 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:37.989540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97adc-eba1-4f41-b70f-ef840d2134d7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.052386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.053041 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.053331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.053608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.054258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.056275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.146051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 823/1672] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:37 2022] POST /identity/v3/users => generated 303 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:38.150083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.160682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5856-7ea7-4779-8fce-001b240248f7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.165418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.166057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.172442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.172442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.172442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.172442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.183371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 850/1673] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:38.185166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.201023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89900-700b-4a79-aa22-1510e4474f2f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.201659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.202342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.202659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.202937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] PATH_INFO: `/v3/projects/0e0a2b269c5344c78caba512f3b5012d/users/08c5d19f2548405ca7ea0018cbd070a3/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.219015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0a2b269c5344c78caba512f3b5012d, user_id=08c5d19f2548405ca7ea0018cbd070a3, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.221708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.330005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 824/1674] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:38 2022] PUT /identity/v3/projects/0e0a2b269c5344c78caba512f3b5012d/users/08c5d19f2548405ca7ea0018cbd070a3/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:38.334113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.345712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.347882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02a8f0-439e-4892-b5bc-ee7938be3b9a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.433609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c88ff9-4e05-49b0-84ec-c6a63d506154 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.442559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.443297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.443659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.443943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] PATH_INFO: `/v3/users/24fc111e6b0447b39eaabb2c387ed7b1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.445137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fc111e6b0447b39eaabb2c387ed7b1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.447124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.449168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.450082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.450535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.450939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.451777 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.453500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.463530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 851/1675] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:38.468466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.478902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c479d-e431-4cb4-aaad-bbbe4c378ed3 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.486495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.487478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.487597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.488088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] PATH_INFO: `/v3/projects/0e0a2b269c5344c78caba512f3b5012d/users/08c5d19f2548405ca7ea0018cbd070a3/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.490262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0a2b269c5344c78caba512f3b5012d, user_id=08c5d19f2548405ca7ea0018cbd070a3, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.493276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef14d7-c205-41f5-a1e5-9d0c5f0a0a30 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.505290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 852/1676] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:38 2022] PUT /identity/v3/projects/0e0a2b269c5344c78caba512f3b5012d/users/08c5d19f2548405ca7ea0018cbd070a3/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:38.515665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d96e8-e717-4b83-b5a4-2e331e899ea7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.515985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d96e8-e717-4b83-b5a4-2e331e899ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.516222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d96e8-e717-4b83-b5a4-2e331e899ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.549965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-da7d96e8-e717-4b83-b5a4-2e331e899ea7 None None] MFA Rules not processed for user `04d43b47c7064ba09f0f3d98eac0574b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:38.551264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-da7d96e8-e717-4b83-b5a4-2e331e899ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.574695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:38.612210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fc111e6b0447b39eaabb2c387ed7b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:38.613097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fc111e6b0447b39eaabb2c387ed7b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:38.621682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fc111e6b0447b39eaabb2c387ed7b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:38.645747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fc111e6b0447b39eaabb2c387ed7b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:38.654838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 853/1677] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:38.656840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee53d-3ae9-44cc-9797-1931c5cf5b14 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.656947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee53d-3ae9-44cc-9797-1931c5cf5b14 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.657247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee53d-3ae9-44cc-9797-1931c5cf5b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.691831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-925ee53d-3ae9-44cc-9797-1931c5cf5b14 None None] MFA Rules not processed for user `08c5d19f2548405ca7ea0018cbd070a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:38.693296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-925ee53d-3ae9-44cc-9797-1931c5cf5b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.716362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fc111e6b0447b39eaabb2c387ed7b1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:38.723335 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 825/1678] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:38 2022] DELETE /identity/v3/users/24fc111e6b0447b39eaabb2c387ed7b1 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:38.724218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.727068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.784197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 854/1679] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:38.785535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.785828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.786110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.792152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc3723-8d7b-4200-9123-bd57e9094c42 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.796643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.797292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.797575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.797852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.799656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.807490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:38.809202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.860721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.862209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.868390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 855/1680] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:38.869234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.870567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.872970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 826/1681] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:38.880123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:38.885388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:38.895273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2e9e-97ee-44ca-b285-ca15b193c180 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.899336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.900055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.900768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.900768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.945099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6f1ca-8e86-43e4-a24c-6ace4384d050 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:38.949758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:38.950687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:38.951548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:38.951833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:38.961756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.963316 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.968585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:38.969970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:38.974081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 827/1682] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:38.981529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 856/1683] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:38.995554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:39.130619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:39.133121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f03ff-10ec-4ae0-835a-27652f4dc38c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:39.136784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:39.137388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:39.137671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:39.137948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:39.139815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:39.202899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:39.204376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:39.215139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 828/1684] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:39.404240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:39.406251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:39.428127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8024c80-6a4a-42c7-9f13-802bed28b1bb service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:39.434051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:39.434494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:39.434811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:39.435088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] PATH_INFO: `/v3/projects/a1540757eff14b6fb6b9cbedb29da293` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:39.436473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1540757eff14b6fb6b9cbedb29da293)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:39.438468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:39.580362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1540757eff14b6fb6b9cbedb29da293', 'request_id': 'req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:39.586396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:39.587665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1540757eff14b6fb6b9cbedb29da293', 'request_id': 'req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:39.687947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1540757eff14b6fb6b9cbedb29da293', 'request_id': 'req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:39.700996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] The token cache is being invalidate because project a1540757eff14b6fb6b9cbedb29da293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:39.701285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:39.701928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:39.703459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 857/1685] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:39 2022] DELETE /identity/v3/projects/a1540757eff14b6fb6b9cbedb29da293 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:39.707429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:39.718784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a9c77-804f-4e1b-8312-49744825446e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:39.775527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:39.776231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:39.776509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:39.776781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] PATH_INFO: `/v3/users/a0db236508fd44eab626c07a659bc576` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:39.778773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0db236508fd44eab626c07a659bc576)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:39.781095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:40.009630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0db236508fd44eab626c07a659bc576'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.010313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0db236508fd44eab626c07a659bc576'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.018323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0db236508fd44eab626c07a659bc576'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.030451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0db236508fd44eab626c07a659bc576'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.121671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-eca3c89f-de5a-430d-8152-42aac2dc80e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0db236508fd44eab626c07a659bc576'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.128259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 829/1686] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:39 2022] DELETE /identity/v3/users/a0db236508fd44eab626c07a659bc576 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:40.517210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:40.673784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:40.674267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:40.688674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1b9f6-2f02-4aec-b4a3-3a8fabccb772 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:40.758145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:40.758812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:40.759096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:40.759377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] PATH_INFO: `/v3/projects/19768d3c9d044003a54961db4bbff213` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:40.760524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19768d3c9d044003a54961db4bbff213)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:40.762469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:40.859132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19768d3c9d044003a54961db4bbff213', 'request_id': 'req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.864974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19768d3c9d044003a54961db4bbff213', 'request_id': 'req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.955217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19768d3c9d044003a54961db4bbff213', 'request_id': 'req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.969172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] The token cache is being invalidate because project 19768d3c9d044003a54961db4bbff213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:40.969464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.970296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ffaad6a2-0e4a-40ea-a6f2-5ccd06286f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:40.972338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 858/1687] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:40 2022] DELETE /identity/v3/projects/19768d3c9d044003a54961db4bbff213 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:40.978259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:40.979003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:40.979215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.001493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:41.002732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.041756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:41.065405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 830/1688] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:41.073323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.073613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.073891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.094938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:41.096158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.104461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 859/1689] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:41.109356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.119101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547f362-a4b1-4949-8c4d-bcbc4d54b52a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.124905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.125673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.125957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.126251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.126928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:41.127234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.128822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.146357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 831/1690] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:41.149649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.159438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c391c-87cc-4de0-8529-915c5e7e8955 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.164591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.165208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.165545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.165818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.166470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.171083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.259758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 860/1691] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:41 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:41.263985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.274741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc20f5b-e919-4794-801a-f913934e0434 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.333747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.334363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.334687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.334967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.336264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.337680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.454054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 832/1692] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:41 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:41.457945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.468488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9ed3-734f-4a0d-93fc-d04760f1e315 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.473541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.474186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.474469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.474769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.475413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.481478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.487856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 861/1693] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:41.491755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.501673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34207a-99eb-4818-b79e-d0b64391bf38 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.509046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.509046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.509046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.509046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.509408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.509693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.520410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 833/1694] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:41.524323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.535363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d64493-1126-436f-a302-3764bcab40ba admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.540239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.540899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.541188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.541460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] PATH_INFO: `/v3/projects/f739f9e94ae145c887bbb3ecf8919b19/users/1f9be369a0e549aebdb15a310b4eec60/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.556989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f739f9e94ae145c887bbb3ecf8919b19, user_id=1f9be369a0e549aebdb15a310b4eec60, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.559732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51ec2b-7aeb-4cf8-826b-55dda81a5cb2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.572122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.573797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.621188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:41.625387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 862/1695] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:41 2022] PUT /identity/v3/projects/f739f9e94ae145c887bbb3ecf8919b19/users/1f9be369a0e549aebdb15a310b4eec60/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:41.636153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575279e5-2d2c-484e-9df2-e684c7be9e13 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.636596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.636953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.637296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.665106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] MFA Rules not processed for user `1f9be369a0e549aebdb15a310b4eec60`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:41.666343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.686642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.687258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.687544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.687822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.689630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.700148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:41.742542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 863/1696] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:41.754717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:41.756606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:41.760718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.762207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.777292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 834/1697] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:41.869138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518eb7-a67f-4f69-b1c2-7b63b6dfe29c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:41.873431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:41.874036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:41.874315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:41.874615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:41.884499 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:41.886072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:41.896061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 864/1698] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:42.037168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:42.322417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:42.500095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:42.848174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:42.850351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:42.860416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bb58-cf55-4477-a2aa-dd4cf0b3eb4b service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:42.864071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:42.864664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:42.865020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:42.865196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:42.867084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:42.939431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:42.940872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def29eb-4bcf-46bd-972d-075cf9789ad3 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:42.951087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 835/1699] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:42.962866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:43.083002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:43.711565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:43.945731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:44.529012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:44.700351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.065198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.099766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.299806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.374195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.510042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.595621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.654539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.796941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.829916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:45.922396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:50.800080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:50.803554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:50.812245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:50.920837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ee9d1-545a-45d8-a723-3fdbf283f84c service glance] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:50.933373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:50.934044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:50.934326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:50.934689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] PATH_INFO: `/v3/users/7423f26ebc1f451ca26bdffa7b67585c` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:50.938883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7423f26ebc1f451ca26bdffa7b67585c)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:50.944449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:51.229300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7423f26ebc1f451ca26bdffa7b67585c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:51.241583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7423f26ebc1f451ca26bdffa7b67585c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:51.261882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7423f26ebc1f451ca26bdffa7b67585c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:51.262764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7423f26ebc1f451ca26bdffa7b67585c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:51.361187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7423f26ebc1f451ca26bdffa7b67585c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:51.378737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:51.379299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:51.379299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:51.380380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 865/1700] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:50 2022] DELETE /identity/v3/users/7423f26ebc1f451ca26bdffa7b67585c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:51.409497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:51.411497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:51.493884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 836/1701] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:51.511740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:51.514347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:51.619341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b35b-9dcf-4819-b8ac-d9e9e2f56664 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:51.624175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:51.624949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:51.625508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:51.625797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:51.650288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:51.655392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:51.666603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 866/1702] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:51.760590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:51.762220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:51.781259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bcb1d-070b-4dd8-8208-77c38a7111ec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:51.792772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:51.793436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:51.793715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:51.794063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] PATH_INFO: `/v3/users/2627eb839af34a3f9e6b4b7234caf44f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:51.796119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2627eb839af34a3f9e6b4b7234caf44f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:51.798053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:52.079425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2627eb839af34a3f9e6b4b7234caf44f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.080098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2627eb839af34a3f9e6b4b7234caf44f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.087968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2627eb839af34a3f9e6b4b7234caf44f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.101318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2627eb839af34a3f9e6b4b7234caf44f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.168106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2834fa4-7240-4064-8f88-043c51c1704f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2627eb839af34a3f9e6b4b7234caf44f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.177727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:52.179186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:52.197227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 837/1703] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:51 2022] DELETE /identity/v3/users/2627eb839af34a3f9e6b4b7234caf44f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:52.198363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5432617-3c18-458e-b97d-3bbddbe39bb6 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:52.199978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:52.200771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:52.201280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:52.235233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:52.242724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:52.246558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:52.270810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:52.271541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:52.271892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:52.272336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] PATH_INFO: `/v3/projects/fc7f3a2502234199bb98921d65a826ff` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:52.273556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7f3a2502234199bb98921d65a826ff)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:52.275678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:52.307584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:52.317940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 838/1704] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:52.328738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:52.330209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:52.393878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd67fe-5ba9-42c6-abe1-5dc414de93da None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:52.398023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7f3a2502234199bb98921d65a826ff', 'request_id': 'req-93de9ac7-cb05-448d-81e4-116c8d124b4e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.403786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7f3a2502234199bb98921d65a826ff', 'request_id': 'req-93de9ac7-cb05-448d-81e4-116c8d124b4e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.447820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:52.448440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:52.448733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:52.449175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:52.487705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7f3a2502234199bb98921d65a826ff', 'request_id': 'req-93de9ac7-cb05-448d-81e4-116c8d124b4e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.499681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] The token cache is being invalidate because project fc7f3a2502234199bb98921d65a826ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:52.499972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.500617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.502308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 867/1705] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:52 2022] DELETE /identity/v3/projects/fc7f3a2502234199bb98921d65a826ff => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:52.506815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:52.508193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:52.511727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:52.513714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:52.521308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de9ac7-cb05-448d-81e4-116c8d124b4e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:52.523727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 839/1706] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:52.531957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:52.532550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:52.532837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:52.533112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] PATH_INFO: `/v3/users/08c5d19f2548405ca7ea0018cbd070a3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:52.534245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c5d19f2548405ca7ea0018cbd070a3)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:52.536202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:52.682473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c5d19f2548405ca7ea0018cbd070a3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.691575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c5d19f2548405ca7ea0018cbd070a3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.713030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c5d19f2548405ca7ea0018cbd070a3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.713030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c5d19f2548405ca7ea0018cbd070a3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.789338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c5d19f2548405ca7ea0018cbd070a3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:52.797414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 868/1707] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:52 2022] DELETE /identity/v3/users/08c5d19f2548405ca7ea0018cbd070a3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:52.937387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:52.937688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:52.955550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3964f6-1ba9-4cd3-a94e-e12cc1d59aee service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:53.035931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:53.036431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:53.036709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:53.036980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] PATH_INFO: `/v3/projects/5cdb28c69bb84c5a8bee83efb2feaf9e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:53.038246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cdb28c69bb84c5a8bee83efb2feaf9e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:53.040221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:53.047400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:53.166646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cdb28c69bb84c5a8bee83efb2feaf9e', 'request_id': 'req-09de525c-76ec-4d13-bd3b-8cf40ac41df8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.172620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cdb28c69bb84c5a8bee83efb2feaf9e', 'request_id': 'req-09de525c-76ec-4d13-bd3b-8cf40ac41df8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.642189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cdb28c69bb84c5a8bee83efb2feaf9e', 'request_id': 'req-09de525c-76ec-4d13-bd3b-8cf40ac41df8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.654004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] The token cache is being invalidate because project 5cdb28c69bb84c5a8bee83efb2feaf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:53.654289 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.654934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.656411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 840/1708] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:52 2022] DELETE /identity/v3/projects/5cdb28c69bb84c5a8bee83efb2feaf9e => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:53.660438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:53.661822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:53.680560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f249c48-8c50-4f30-9299-26e894fe3ae0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:53.681768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:53.683402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:53.708902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:53.740949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:53.741589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:53.741871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:53.742203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] PATH_INFO: `/v3/users/9949ff8d2ab14ebd8a17b9560ced00e7` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:53.743378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9949ff8d2ab14ebd8a17b9560ced00e7)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:53.748094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:53.755158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de525c-76ec-4d13-bd3b-8cf40ac41df8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:53.760059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:53.760682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:53.760959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:53.761230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] PATH_INFO: `/v3/projects/0e0a2b269c5344c78caba512f3b5012d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:53.762337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0a2b269c5344c78caba512f3b5012d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:53.764803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:53.944910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0a2b269c5344c78caba512f3b5012d', 'request_id': 'req-aac1870d-320a-4eb7-b068-d24ebffbbf2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.953212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0a2b269c5344c78caba512f3b5012d', 'request_id': 'req-aac1870d-320a-4eb7-b068-d24ebffbbf2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.955656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9949ff8d2ab14ebd8a17b9560ced00e7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.966211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9949ff8d2ab14ebd8a17b9560ced00e7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.979074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9949ff8d2ab14ebd8a17b9560ced00e7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:53.979944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9949ff8d2ab14ebd8a17b9560ced00e7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.023739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0a2b269c5344c78caba512f3b5012d', 'request_id': 'req-aac1870d-320a-4eb7-b068-d24ebffbbf2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.036519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] The token cache is being invalidate because project 0e0a2b269c5344c78caba512f3b5012d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:54.036805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.037444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aac1870d-320a-4eb7-b068-d24ebffbbf2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.039056 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 841/1709] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:53 2022] DELETE /identity/v3/projects/0e0a2b269c5344c78caba512f3b5012d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:54.043640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.043935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.044242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.063449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:54.064640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.084043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:54.086102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5b2ea78-a436-4638-a198-feb8baa68641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9949ff8d2ab14ebd8a17b9560ced00e7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.093168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 869/1710] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:53 2022] DELETE /identity/v3/users/9949ff8d2ab14ebd8a17b9560ced00e7 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:54.126629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 842/1711] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:54.134410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.134723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.135385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.156756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:54.157967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.221519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 870/1712] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:54.225856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.235474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdc11c-ffd3-4e9c-9895-7e331f608a8d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.240867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.241473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.241755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.242029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.242721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:54.243054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.244667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.256835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 843/1713] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:54.261453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.271381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482bcad-3004-4f28-b74e-6b53e180bd08 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.276161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.276765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.277306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.277306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.277938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.281105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.396620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 871/1714] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/projects => generated 399 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:54.397813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.408303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45080cbd-a5c0-43b4-8e51-37b3a5be2316 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.465869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.466522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.466813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.467083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.467733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.469624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.483471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.485178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.498380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4569-b109-4c0b-91d3-595b6e0c9387 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.503206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.503872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.504152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.504424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] PATH_INFO: `/v3/projects/25e12b2be27d46e3ac1ea01a75a52941` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.505545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e12b2be27d46e3ac1ea01a75a52941)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.512277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.532540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:54.545524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 844/1715] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:54.549550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.559139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15159f-4dbe-4867-a98d-4a82bb68c47d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.564533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.565121 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.565407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.565681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.566357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.567966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.578659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 845/1716] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:54.582468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.592283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4938-40d3-462a-8482-441c0852f63c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.597242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e12b2be27d46e3ac1ea01a75a52941', 'request_id': 'req-0e3ad561-e0f9-432c-a246-d59d434a1ef3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.602821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e12b2be27d46e3ac1ea01a75a52941', 'request_id': 'req-0e3ad561-e0f9-432c-a246-d59d434a1ef3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.650096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.650735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.651020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.651293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.651933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.653526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.668099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 846/1717] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:54.669961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.680063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9f97-9f57-4d84-a86c-57a8a37034a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.684762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.686338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.686673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.686946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] PATH_INFO: `/v3/projects/cc20cae19a0248afab56dd33b9ea5546/users/9a1cc197e081405faae388617ea274de/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.693253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e12b2be27d46e3ac1ea01a75a52941', 'request_id': 'req-0e3ad561-e0f9-432c-a246-d59d434a1ef3'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.702551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc20cae19a0248afab56dd33b9ea5546, user_id=9a1cc197e081405faae388617ea274de, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.706532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc55b9-c041-40c2-a8b5-7a1c3f32b4c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.708192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] The token cache is being invalidate because project 25e12b2be27d46e3ac1ea01a75a52941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:54.708641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.709639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0e3ad561-e0f9-432c-a246-d59d434a1ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:54.711232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 872/1718] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:54 2022] DELETE /identity/v3/projects/25e12b2be27d46e3ac1ea01a75a52941 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:54.715970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc875b-4ce2-461e-b40c-07ced0892c73 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.716298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc875b-4ce2-461e-b40c-07ced0892c73 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.716580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc875b-4ce2-461e-b40c-07ced0892c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.737258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d2bc875b-4ce2-461e-b40c-07ced0892c73 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:54.738558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d2bc875b-4ce2-461e-b40c-07ced0892c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.747495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 873/1719] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:54.755575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.755870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.756152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.775795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:54.777013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.779973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 847/1720] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:54 2022] PUT /identity/v3/projects/cc20cae19a0248afab56dd33b9ea5546/users/9a1cc197e081405faae388617ea274de/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:54.791106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.791392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.791670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.799264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:54.822567 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] MFA Rules not processed for user `9a1cc197e081405faae388617ea274de`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:54.823938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.860822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 874/1721] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:54.864917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.876781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22248bfd-71dd-4b78-a586-46fd19798384 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.881985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.882877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.883272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.883549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.884193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:54.884613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.886202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.901417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 875/1722] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:35:54.904718 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 848/1723] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:54.907853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.909824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:54.909957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:54.920902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2115b-fa46-410e-a755-b52e1891c705 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.924119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb899-36a9-426d-aa1a-71795939a50a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:54.926270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.926887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.927166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.927441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.928071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.928970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:54.929587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:54.929877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:54.929972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:54.930157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:54.930818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:54.932723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.020708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 849/1724] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:55.026160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.053291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 876/1725] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:54 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:55.057866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.099383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c197a-9772-4a30-aaf6-673e0c5fa9d1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.129239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c27360-481e-4e60-972b-578c0ba3d377 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.134703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.135313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.135588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.135896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.136524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.138551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.162118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.162887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.163246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.163611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.164307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.166306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.247074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:55.256093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 877/1726] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:55 2022] POST /identity/v3/users => generated 306 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:55.257214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.266644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad694a4-de10-425e-b050-ce854b1e745f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.271609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.272191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.272473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.272748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.273465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.275265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.284285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 850/1727] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:55 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:35:55.285450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.285813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 878/1728] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:55.289712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.296130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e5189-0748-45a4-830d-cc5abe6a60ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.300264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e42b-e46b-49e7-8a1f-91584e8584ce admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.301857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.301857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.301857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.302156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.302648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.304221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.305246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.305651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:55.305896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.306176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.306448 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] PATH_INFO: `/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9/users/f10578c5d77f4957adf11e659c7881fd/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.315459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 851/1729] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:55.319421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.322091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c75f76f296445782ff6fbfb34b0cc9, user_id=f10578c5d77f4957adf11e659c7881fd, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.324884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.329717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94c742-47b2-4d5e-8a7a-46794a14f4a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.334407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.335037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.335317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.335589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.336212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.337799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.348973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 852/1730] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:55.352893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.362103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab267a28-9d53-45eb-8b26-3c98d48b3b4e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.367029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.367662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.367941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.370776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] PATH_INFO: `/v3/projects/5c9979da525742f8b6ccfda027a399eb/users/9ff5d14dc3f74d8791c4c0b86a891620/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.383620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c9979da525742f8b6ccfda027a399eb, user_id=9ff5d14dc3f74d8791c4c0b86a891620, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.386212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd27b5-d94e-4ea6-a1ee-d5fda7d37001 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.419083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 879/1731] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:55 2022] PUT /identity/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9/users/f10578c5d77f4957adf11e659c7881fd/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:55.423007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.464852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 853/1732] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:55 2022] PUT /identity/v3/projects/5c9979da525742f8b6ccfda027a399eb/users/9ff5d14dc3f74d8791c4c0b86a891620/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:55.476862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.477149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.477430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.493518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe74115-2989-4d79-822b-7eae49bf0a6f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.505671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] MFA Rules not processed for user `9ff5d14dc3f74d8791c4c0b86a891620`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:55.506955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:55.555169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.555782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.556065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.556360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.556982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.558613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.568838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 880/1733] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:55.573208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.574813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 854/1734] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:55.586335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062afec-ceef-4ce2-b3df-3b31ffc7aced admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.586954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.588114 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.588425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:55.588761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.589075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.589359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] PATH_INFO: `/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9/users/f10578c5d77f4957adf11e659c7881fd/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.591789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c75f76f296445782ff6fbfb34b0cc9, user_id=f10578c5d77f4957adf11e659c7881fd, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.594702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f258c-f0c6-4d7e-8370-768dc3e1e171 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.605835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 881/1735] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:55 2022] PUT /identity/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9/users/f10578c5d77f4957adf11e659c7881fd/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:35:55.616433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.616719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.617002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.645637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] MFA Rules not processed for user `f10578c5d77f4957adf11e659c7881fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:55.646562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdde55-68dd-4c60-8f6d-b1293a272ffd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.646831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:55.688205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:55.698664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.706785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.707693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.708015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.717952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 882/1736] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:55.729240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:55.767096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.768525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.777240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 855/1737] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:55.796166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.798133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:55.806490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101011c-1410-4b63-ada9-5eab4a860081 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.811447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.812139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.812415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.812687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.822136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.824129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:55.836032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 856/1738] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:55.862101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8669b6f-2e2d-475b-b6ea-c060128727c7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:55.866746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:55.867385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:55.867666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:55.867941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] PATH_INFO: `/v3/users/9ff5d14dc3f74d8791c4c0b86a891620` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:55.869217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff5d14dc3f74d8791c4c0b86a891620)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:55.871190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:56.068394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff5d14dc3f74d8791c4c0b86a891620'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.076014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff5d14dc3f74d8791c4c0b86a891620'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.088315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff5d14dc3f74d8791c4c0b86a891620'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.089041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff5d14dc3f74d8791c4c0b86a891620'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.111230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:56.119570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c209c-90c3-476c-8718-25e05e301f4a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:56.123839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:56.124461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:56.124757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:56.125007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:56.126768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:56.127843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f0192954-43f3-497d-9b90-7bc38fc265d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff5d14dc3f74d8791c4c0b86a891620'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.134426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 883/1739] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:35:55 2022] DELETE /identity/v3/users/9ff5d14dc3f74d8791c4c0b86a891620 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:56.138630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:56.138922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:56.139223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:56.158637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:56.159767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:56.189902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:56.191536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:56.201347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 857/1740] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:35:56.221562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:56.236997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 884/1741] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:35:56.246907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:56.249550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:56.327051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99839a07-dc7f-46ac-84b7-4332e495165f service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:56.378464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:56.379100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:56.379394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:56.379649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:56.389301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:56.390805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0632784-ecef-4fca-a6a9-8eb1a0fc2be7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:56.400314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 858/1742] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:35:56.785808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:56.788000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:56.803003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9072fd-ec6f-4f0a-b78d-0c01b0960fea None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:56.819973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:56.820827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:56.821329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:56.822313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] PATH_INFO: `/v3/projects/5c9979da525742f8b6ccfda027a399eb` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:56.824083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c9979da525742f8b6ccfda027a399eb)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:56.827984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:56.932074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c9979da525742f8b6ccfda027a399eb', 'request_id': 'req-186518b4-0828-42e9-b9e8-b9e2a26ffb67'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:56.937953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c9979da525742f8b6ccfda027a399eb', 'request_id': 'req-186518b4-0828-42e9-b9e8-b9e2a26ffb67'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:57.014451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c9979da525742f8b6ccfda027a399eb', 'request_id': 'req-186518b4-0828-42e9-b9e8-b9e2a26ffb67'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:57.026438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] The token cache is being invalidate because project 5c9979da525742f8b6ccfda027a399eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:35:57.026764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:57.027420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-186518b4-0828-42e9-b9e8-b9e2a26ffb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:35:57.029555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 885/1743] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:35:56 2022] DELETE /identity/v3/projects/5c9979da525742f8b6ccfda027a399eb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:57.034370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.039698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.039698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.043376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:57.053981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:57.055269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:57.115673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 859/1744] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:57.122973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.123306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.123714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.144127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:57.145318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:57.153511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 886/1745] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:57.158404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.168262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da4102-f044-4cd5-8c28-eebe0aa0db58 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.172848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.173443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.173719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.173993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.174679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:35:57.175009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.176577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.189019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 860/1746] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:35:57.196833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.206886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abdefa-9f89-4314-9c08-dda2e73e6d08 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.211568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.212272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.212555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.212830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.213455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.215472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.346450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 887/1747] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:35:57 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:57.350586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.360660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dc67e-b8c1-4a57-a58d-76d15d465d83 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.426528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.427153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.427438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.427715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.428364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.430433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.530654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 861/1748] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:35:57 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:35:57.531812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.544097 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a77aba-4abb-4e4c-8d5e-cb27a0a716f7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.550441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.552206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.552206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.552206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.552485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.554030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.565324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 888/1749] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:35:57.573498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.584276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadfb37-be58-48f6-97ea-a59609eb1da7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.589016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.589722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.590076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.590427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.591209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.592863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353c68e-82ab-4e2f-bf38-e0d72009341c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.604294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 862/1750] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:35:57.608714 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.619160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb3b51-0bf3-45b2-8600-2a5702ce3726 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.624434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.625085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.625389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.625662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] PATH_INFO: `/v3/projects/304d441864be4cc29cb830edbd9a5af3/users/5c57cd45b190493faa03afef97356b82/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.640072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] RBAC: Authorizing `identity:create_grant(project_id=304d441864be4cc29cb830edbd9a5af3, user_id=5c57cd45b190493faa03afef97356b82, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.642767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.703001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 889/1751] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:35:57 2022] PUT /identity/v3/projects/304d441864be4cc29cb830edbd9a5af3/users/5c57cd45b190493faa03afef97356b82/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:35:57.720991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.721347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.721634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.750991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] MFA Rules not processed for user `5c57cd45b190493faa03afef97356b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:35:57.752120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:57.816567 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:35:57.820250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 863/1752] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:35:57.833357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:35:57.833357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:35:57.890399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f01864-38d9-46fa-9fbf-0d10706f458b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:35:57.894011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:35:57.894665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:35:57.894945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:35:57.895247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:35:57.903400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:35:57.905441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46591610-7b1d-43b0-8160-d185890f0f8f service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:35:57.913959 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 890/1753] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:07.910048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:19.706344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:19.707929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:19.792111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199ee62-ece6-45f2-89e6-6f1f83a5c586 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:19.796701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:19.797338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:19.797620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:19.797899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] PATH_INFO: `/v3/users/9a1cc197e081405faae388617ea274de` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:19.799621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a1cc197e081405faae388617ea274de)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:19.801538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:20.029533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a1cc197e081405faae388617ea274de'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.030194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a1cc197e081405faae388617ea274de'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.038181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a1cc197e081405faae388617ea274de'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.061563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a1cc197e081405faae388617ea274de'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.161726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a1cc197e081405faae388617ea274de'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.169336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 864/1754] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:19 2022] DELETE /identity/v3/users/9a1cc197e081405faae388617ea274de => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:20.174107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:20.174585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:20.174677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:20.193715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:20.194866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:20.252962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 891/1755] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:20.262525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:20.263941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:20.322987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb37f4-a1cd-4288-af94-86b941465ff2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:20.326801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:20.327407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:20.327685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:20.327996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:20.337624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:20.340592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:20.348371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 865/1756] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:20.740434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:20.741913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:20.754949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0542-75b3-427f-bb12-2a7e9642be68 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:20.759641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:20.760341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:20.760619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:20.760893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] PATH_INFO: `/v3/projects/cc20cae19a0248afab56dd33b9ea5546` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:20.762062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc20cae19a0248afab56dd33b9ea5546)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:20.764043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:20.857631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc20cae19a0248afab56dd33b9ea5546', 'request_id': 'req-a763e5a5-be08-40c8-b8c5-d565678ed329'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.864122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc20cae19a0248afab56dd33b9ea5546', 'request_id': 'req-a763e5a5-be08-40c8-b8c5-d565678ed329'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.920973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc20cae19a0248afab56dd33b9ea5546', 'request_id': 'req-a763e5a5-be08-40c8-b8c5-d565678ed329'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.933754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] The token cache is being invalidate because project cc20cae19a0248afab56dd33b9ea5546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:20.934145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.935156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:20.936718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 892/1757] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:20 2022] DELETE /identity/v3/projects/cc20cae19a0248afab56dd33b9ea5546 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:20.941478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:20.951044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5661e-b595-4342-a5a0-e523c17d3c31 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:20.968956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:21.020721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:21.021417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:21.021697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:21.021973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] PATH_INFO: `/v3/users/f10578c5d77f4957adf11e659c7881fd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:21.023334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10578c5d77f4957adf11e659c7881fd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:21.025285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:21.221333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10578c5d77f4957adf11e659c7881fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.221985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10578c5d77f4957adf11e659c7881fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.228997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10578c5d77f4957adf11e659c7881fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.241873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10578c5d77f4957adf11e659c7881fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.292310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fbda1c28-f1c0-4631-ac6c-fc25e6c45f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10578c5d77f4957adf11e659c7881fd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.298278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 866/1758] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:20 2022] DELETE /identity/v3/users/f10578c5d77f4957adf11e659c7881fd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:21.693754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:21.695270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:21.709432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763e5a5-be08-40c8-b8c5-d565678ed329 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:21.770332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:21.771100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:21.771380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:21.771656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] PATH_INFO: `/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:21.772770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c75f76f296445782ff6fbfb34b0cc9)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:21.774857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:21.865174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:21.881142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c75f76f296445782ff6fbfb34b0cc9', 'request_id': 'req-e70049b9-7dce-49a8-88ed-d86778a79746'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.886721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c75f76f296445782ff6fbfb34b0cc9', 'request_id': 'req-e70049b9-7dce-49a8-88ed-d86778a79746'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.972138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c75f76f296445782ff6fbfb34b0cc9', 'request_id': 'req-e70049b9-7dce-49a8-88ed-d86778a79746'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.983581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] The token cache is being invalidate because project e4c75f76f296445782ff6fbfb34b0cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:21.983885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.984515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e70049b9-7dce-49a8-88ed-d86778a79746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:21.986033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 893/1759] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:21 2022] DELETE /identity/v3/projects/e4c75f76f296445782ff6fbfb34b0cc9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:21.991000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:21.991453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:21.991587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.012519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:22.013719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:22.075230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 867/1760] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:22.084649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.084995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.085214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.104630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:22.105821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:22.114342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 894/1761] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:22.118352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.128842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc1eaf-3cf4-434d-910f-b20e0570f65e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.133607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.134252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.135197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.135197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.135470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:22.135772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.137380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.155706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 868/1762] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:22.157262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.166973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd303aeb-fd5d-4be2-b6ef-4e7d010a31a9 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.173533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.174818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.175195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.175586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.176325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.178330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.291855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 895/1763] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:22.293499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.303942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ad41-5cbc-4666-a9e6-7f844033c4d9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.331083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:22.364285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.364901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.365341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.365621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.366253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.368202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.469576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 869/1764] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:22.473765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.484154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777406b0-500d-4e0c-ae8d-afe465693f43 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.489227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.489820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.490131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.490401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.492365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.492699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.503461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 896/1765] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:22.507226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.516876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298db2ff-d905-46bf-9023-fb85b1685061 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.521459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.522040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.522314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.522621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.523252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.524816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61dd232-37ba-4c36-b7b8-a4480fbf51f6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.535390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 870/1766] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:22.539280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.549868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7d774-4b0e-4ae0-b157-77960f861579 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.554612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.555259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.555541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.555929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] PATH_INFO: `/v3/projects/b2ad09764d1e4132ac15228c3bbe456c/users/1cc304b2c9fc4ee8bb78f1730872d8f0/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.570361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ad09764d1e4132ac15228c3bbe456c, user_id=1cc304b2c9fc4ee8bb78f1730872d8f0, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.573032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.635137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 897/1767] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:22 2022] PUT /identity/v3/projects/b2ad09764d1e4132ac15228c3bbe456c/users/1cc304b2c9fc4ee8bb78f1730872d8f0/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:22.645714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.645997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.646270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.674666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] MFA Rules not processed for user `1cc304b2c9fc4ee8bb78f1730872d8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:22.675913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:22.740680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 871/1768] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:22.745099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.805771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d40029-e03c-4436-8111-1e60f54dc787 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.810814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.811420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.811705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.811980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.812609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.814622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:22.904132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 898/1769] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:22.908746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:22.919556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6946cbb-7fbf-43c5-b354-cc8725c8f6c0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:22.977200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:22.977808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:22.978093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:22.978370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:22.979050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:22.980962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.057541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 872/1770] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:22 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:23.061614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.071867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ccf31-0466-497a-9f7c-7f6b37ea8fed admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.076716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.077318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.077602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.077879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.078588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.080759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.092592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 899/1771] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:23.097227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.107099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca47e-589a-4df5-87bd-a476c6c640c2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.111784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.112455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.112729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.113000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] PATH_INFO: `/v3/projects/52a30ca05df44108872bbbcde68b48a0/users/220f9f11b2b846fd9e28acc35771d60f/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.127649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a30ca05df44108872bbbcde68b48a0, user_id=220f9f11b2b846fd9e28acc35771d60f, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.130280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.201044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 873/1772] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:23 2022] PUT /identity/v3/projects/52a30ca05df44108872bbbcde68b48a0/users/220f9f11b2b846fd9e28acc35771d60f/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:23.205554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.215326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d30eda-37b1-439c-841f-903096a9de71 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.235587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:23.277935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.278701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.279080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.279493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.280311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.282126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3b6f7-1a61-47db-8fab-2c99f6c41b82 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.298723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 900/1773] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:23.302982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.308490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:23.314664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc417-c884-4c43-812c-bbb629616ecb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.320025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.320740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.321042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.321291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] PATH_INFO: `/v3/projects/52a30ca05df44108872bbbcde68b48a0/users/220f9f11b2b846fd9e28acc35771d60f/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.323196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a30ca05df44108872bbbcde68b48a0, user_id=220f9f11b2b846fd9e28acc35771d60f, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.325898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.337160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 874/1774] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:23 2022] PUT /identity/v3/projects/52a30ca05df44108872bbbcde68b48a0/users/220f9f11b2b846fd9e28acc35771d60f/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:23.347289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.347586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.347892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.375973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] MFA Rules not processed for user `220f9f11b2b846fd9e28acc35771d60f`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:23.377726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:23.450984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 901/1775] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:23.462547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.464270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:23.524664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10d29b-80e2-46f3-b2ee-c49aad9ba431 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.528516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.529132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.529416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.529721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.539368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.540808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.551861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 875/1776] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:23.652739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:23.662119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e312f-a0ef-45ea-867e-800abb9abc1e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:23.665709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:23.666303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:23.666616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:23.666892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:23.723113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:23.724574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c65ff-285a-4862-b758-7a71f7d8afa3 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:23.733538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 902/1777] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:33.547274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:33.729390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=80 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:35.453355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:35.524307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51991d80-0a81-4879-95c5-9ba704f63823 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:35.529125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:35.529775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:35.530190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:35.530473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] PATH_INFO: `/v3/users/1cc304b2c9fc4ee8bb78f1730872d8f0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:35.538486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc304b2c9fc4ee8bb78f1730872d8f0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:35.538486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:35.775003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc304b2c9fc4ee8bb78f1730872d8f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:35.775686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc304b2c9fc4ee8bb78f1730872d8f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:35.783423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc304b2c9fc4ee8bb78f1730872d8f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:35.795776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc304b2c9fc4ee8bb78f1730872d8f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:35.875676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc304b2c9fc4ee8bb78f1730872d8f0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:35.884975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 876/1778] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:35 2022] DELETE /identity/v3/users/1cc304b2c9fc4ee8bb78f1730872d8f0 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:35.889718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:35.890241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:35.890498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:35.923023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:35.923023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:35.988809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 903/1779] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:35.996538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:35.997958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:36.081220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45849f3b-92e9-4b39-9719-f9ac83de3115 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:36.108734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:36.108734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:36.108734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:36.108734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:36.116723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:36.118226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:36.131442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 877/1780] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:36.636780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:36.638381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:36.652824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c218071-ee68-435b-ad8f-4638b86e1c04 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:36.659746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:36.661434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:36.661434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:36.661434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] PATH_INFO: `/v3/projects/b2ad09764d1e4132ac15228c3bbe456c` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:36.662680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ad09764d1e4132ac15228c3bbe456c)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:36.664986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:36.786063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ad09764d1e4132ac15228c3bbe456c', 'request_id': 'req-31e39382-1d4a-434a-bfbc-adda0adfefa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:36.792439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ad09764d1e4132ac15228c3bbe456c', 'request_id': 'req-31e39382-1d4a-434a-bfbc-adda0adfefa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:36.863895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ad09764d1e4132ac15228c3bbe456c', 'request_id': 'req-31e39382-1d4a-434a-bfbc-adda0adfefa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:36.875651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] The token cache is being invalidate because project b2ad09764d1e4132ac15228c3bbe456c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:36.875937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:36.876554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:36.878141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 904/1781] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:36 2022] DELETE /identity/v3/projects/b2ad09764d1e4132ac15228c3bbe456c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:36.882327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:36.892101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71909d06-615b-4e78-a3a3-99bbe75ea470 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:36.911791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:36.950842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:36.951506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:36.951743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:36.952016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] PATH_INFO: `/v3/users/220f9f11b2b846fd9e28acc35771d60f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:36.953132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220f9f11b2b846fd9e28acc35771d60f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:36.955140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:37.134353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220f9f11b2b846fd9e28acc35771d60f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.135087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220f9f11b2b846fd9e28acc35771d60f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.142063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220f9f11b2b846fd9e28acc35771d60f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.155436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220f9f11b2b846fd9e28acc35771d60f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.229109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1dc68683-3202-4b68-899b-61ca66ae8472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220f9f11b2b846fd9e28acc35771d60f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.235327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 878/1782] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:36 2022] DELETE /identity/v3/users/220f9f11b2b846fd9e28acc35771d60f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:37.733608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:37.735047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:37.761282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e39382-1d4a-434a-bfbc-adda0adfefa0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:37.832888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:37.833529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:37.833808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:37.834115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] PATH_INFO: `/v3/projects/52a30ca05df44108872bbbcde68b48a0` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:37.835253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a30ca05df44108872bbbcde68b48a0)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:37.837191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:37.938408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:37.956201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a30ca05df44108872bbbcde68b48a0', 'request_id': 'req-cddb5c36-b3e1-4558-8714-e2f529ec0676'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:37.962287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a30ca05df44108872bbbcde68b48a0', 'request_id': 'req-cddb5c36-b3e1-4558-8714-e2f529ec0676'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:38.069016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a30ca05df44108872bbbcde68b48a0', 'request_id': 'req-cddb5c36-b3e1-4558-8714-e2f529ec0676'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:38.081217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] The token cache is being invalidate because project 52a30ca05df44108872bbbcde68b48a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:38.081505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:38.082132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cddb5c36-b3e1-4558-8714-e2f529ec0676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:38.083659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 905/1783] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:37 2022] DELETE /identity/v3/projects/52a30ca05df44108872bbbcde68b48a0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:38.088684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.089160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.089254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.109206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:38.110345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:38.170141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 879/1784] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:38.178680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.178969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.182376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.200237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:38.201445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:38.209556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 906/1785] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:38.214096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.224203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf092b-afa9-4690-81ed-f6b8e7cd40a6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.228818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.229418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.230442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.230442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.230661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:38.231027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.232601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.244508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 880/1786] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:38.248755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.258696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c4351-30c3-4b5d-8a7a-8f1ce07981ed None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.264631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.264631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.264631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.264631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.265268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.267226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.345455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 907/1787] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:38.350059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.360706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169220e-1c61-4838-8695-2d0eeba49670 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.387581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:38.417782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.418421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.418763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.419044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.419754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.421671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.487826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 881/1788] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:38.496836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.507778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5d2c3-effc-4954-b664-841badb39c7c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.512996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.513630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.513917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.514192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.514931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.516538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.526927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 908/1789] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:38.530995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.541759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dffdd-c522-48b4-9d34-fffbaca695a3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.546469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.547112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.547384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.547713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.548383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.550680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0228660-1b93-4219-aa49-9c99f7b31f1a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.561008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 882/1790] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:38.565127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.575009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576bf7f-1d34-4d13-bf8c-80ab17ccd400 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.579893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.580535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.580819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.581099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] PATH_INFO: `/v3/projects/3cd063f708ac4673b46ad20b068ca99b/users/409f6cf3f4e34f48a57b4480e3fa32ec/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.597597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd063f708ac4673b46ad20b068ca99b, user_id=409f6cf3f4e34f48a57b4480e3fa32ec, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.600446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.669590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 909/1791] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:38 2022] PUT /identity/v3/projects/3cd063f708ac4673b46ad20b068ca99b/users/409f6cf3f4e34f48a57b4480e3fa32ec/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:38.679823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.680574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.680574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.717116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] MFA Rules not processed for user `409f6cf3f4e34f48a57b4480e3fa32ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:38.719217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:38.791760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 883/1792] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:38.796329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:38.881956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2726784-ab2c-4032-a53f-57e8702f3086 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:38.887265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:38.887966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:38.888319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:38.888600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:38.889244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:38.891250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:38.991007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 910/1793] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:38.995115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.008065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a92646-c67c-4c93-9109-63df906a7a1f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.074531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.075193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.075476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.075753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.076381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.078288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.166798 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 884/1794] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:38 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:39.170909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.181042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d33a-d4cd-4f7e-9b02-b95574bcc4f7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.186237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.187044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.187279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.187581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.188219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.190353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.200821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 911/1795] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:39.205694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.215625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c2305-70ee-443b-849d-b2b4e5aa2af6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.220449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.221092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.221411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.221690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] PATH_INFO: `/v3/projects/975eef09277c47769b75c01d5ab99b21/users/91d2cc1a6f044e1aa3923bfd96cd8bad/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.238549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=975eef09277c47769b75c01d5ab99b21, user_id=91d2cc1a6f044e1aa3923bfd96cd8bad, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.241232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.343185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 885/1796] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:39 2022] PUT /identity/v3/projects/975eef09277c47769b75c01d5ab99b21/users/91d2cc1a6f044e1aa3923bfd96cd8bad/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:39.347067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.356991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d26252-f7a4-4f3f-b82c-e45c9feadc4a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.376482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:39.412552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.413085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.413363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.413634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.414266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.415990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3730a6-5a57-4e14-8c05-9b7e04a5fff6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.425875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 912/1797] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:39.430175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.435956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:39.441806 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7c641-80ca-4c7b-90b8-913d951bef39 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.446426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.447111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.447377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.447649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] PATH_INFO: `/v3/projects/975eef09277c47769b75c01d5ab99b21/users/91d2cc1a6f044e1aa3923bfd96cd8bad/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.449521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] RBAC: Authorizing `identity:create_grant(project_id=975eef09277c47769b75c01d5ab99b21, user_id=91d2cc1a6f044e1aa3923bfd96cd8bad, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.452300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.462212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 886/1798] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:39 2022] PUT /identity/v3/projects/975eef09277c47769b75c01d5ab99b21/users/91d2cc1a6f044e1aa3923bfd96cd8bad/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:39.473136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.473421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.473701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.501728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] MFA Rules not processed for user `91d2cc1a6f044e1aa3923bfd96cd8bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:39.503577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:39.571189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 913/1799] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:39.582690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.584378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:39.641410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfb74-c997-438d-820c-4cd3b0c2720d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.645217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.645834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.646111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.646384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.656011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.657769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.667140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 887/1800] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:39.793581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:39.795110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:39.807808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a03b5-8a2b-432a-ac8b-656ba84aa9fd None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:39.811637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:39.812229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:39.812507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:39.812778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:39.871237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:39.873729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20415d9-5169-4ccf-b8e3-dba47ba28047 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:39.883730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 914/1801] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:40.088277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:40.090534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:40.170794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eaa2-967a-4e9b-bf0b-df0480369e76 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:40.176091 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:40.176970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:40.177258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:40.182173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] PATH_INFO: `/v3/users/d41ccac0df8e4decb4496a000ee78d1c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:40.182173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41ccac0df8e4decb4496a000ee78d1c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:40.182173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:40.427033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41ccac0df8e4decb4496a000ee78d1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.427630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41ccac0df8e4decb4496a000ee78d1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.434934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41ccac0df8e4decb4496a000ee78d1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.447813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41ccac0df8e4decb4496a000ee78d1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.520538 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41ccac0df8e4decb4496a000ee78d1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.526948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 888/1802] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:40 2022] DELETE /identity/v3/users/d41ccac0df8e4decb4496a000ee78d1c => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:40.531389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:40.531684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:40.531965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:40.551334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:40.552537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:40.590731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:40.593152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:40.610323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:40.619639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:40.642267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 915/1803] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:40.653257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:40.654823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:40.677029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21761e-9e9a-423c-81b2-216da3f6739d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:40.681732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:40.682310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:40.682625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:40.682898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] PATH_INFO: `/v3/users/409f6cf3f4e34f48a57b4480e3fa32ec` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:40.684060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=409f6cf3f4e34f48a57b4480e3fa32ec)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:40.686311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:40.725218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6e50a-99ab-4a39-a04e-0c4415755969 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:40.728894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:40.729508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:40.729791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:40.730068 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:40.739552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:40.740971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb67d8-5b43-48bf-a0be-8509c67933cb service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:40.751263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 916/1804] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:40.882937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409f6cf3f4e34f48a57b4480e3fa32ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.883740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409f6cf3f4e34f48a57b4480e3fa32ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.894451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409f6cf3f4e34f48a57b4480e3fa32ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.913370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409f6cf3f4e34f48a57b4480e3fa32ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.978281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409f6cf3f4e34f48a57b4480e3fa32ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:40.985160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 889/1805] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:40 2022] DELETE /identity/v3/users/409f6cf3f4e34f48a57b4480e3fa32ec => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:40.990112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c209d1-32e4-48d8-aa2a-deac5abeddb4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:40.990643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c209d1-32e4-48d8-aa2a-deac5abeddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:40.990757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c209d1-32e4-48d8-aa2a-deac5abeddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.026153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-03c209d1-32e4-48d8-aa2a-deac5abeddb4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:41.027415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-03c209d1-32e4-48d8-aa2a-deac5abeddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:41.110222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 917/1806] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:41.136860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:41.136860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:41.204256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd261-2af7-42c3-80f1-8253b313d04c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:41.208054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:41.208773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:41.209151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:41.209533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.219320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:41.220981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274af583-be19-4e79-841d-035cdac1d3f4 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:41.231870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 890/1807] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:41.390353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:41.390867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:41.391870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.394313 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 918/1808] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:36:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:36:41.404093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:41.404711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:41.405066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.433922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:41.436543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:41.479531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:41.514910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 891/1809] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:41.528732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:41.529041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:41.549446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb60e7-1812-4afd-bff5-89838eb6b923 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:41.555152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:41.556105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:41.556243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:41.556549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.566372 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:41.568505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:41.579619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 919/1810] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:41.685554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:41.687108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:41.705455 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c47d07-d7a0-4c85-9ee8-d7e5c63242b5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:41.710949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:41.711679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:41.712021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:41.712428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] PATH_INFO: `/v3/projects/9feb901e5a114f008dc3d94046ffa99d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:41.713913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9feb901e5a114f008dc3d94046ffa99d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:41.716030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:41.816938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9feb901e5a114f008dc3d94046ffa99d', 'request_id': 'req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:41.837270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9feb901e5a114f008dc3d94046ffa99d', 'request_id': 'req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:41.904788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9feb901e5a114f008dc3d94046ffa99d', 'request_id': 'req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:41.919488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] The token cache is being invalidate because project 9feb901e5a114f008dc3d94046ffa99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:41.919879 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:41.920642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:41.922385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 892/1811] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:41 2022] DELETE /identity/v3/projects/9feb901e5a114f008dc3d94046ffa99d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:41.926689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:41.938711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8ad30-06c5-42a8-b58e-ac022b7a4109 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:42.009374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:42.010700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:42.011128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:42.011564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] PATH_INFO: `/v3/users/e0858fbb3c6e47829e64e0fb96029977` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:42.015931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0858fbb3c6e47829e64e0fb96029977)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:42.020685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:42.040587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:42.042601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:42.045666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:42.058648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5a39a-eceb-40f3-a4ad-a03cb068c2bd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:42.066023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:42.066812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:42.090603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:42.090936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] PATH_INFO: `/v3/projects/3cd063f708ac4673b46ad20b068ca99b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:42.094350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cd063f708ac4673b46ad20b068ca99b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:42.096371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:42.251187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0858fbb3c6e47829e64e0fb96029977'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.253186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cd063f708ac4673b46ad20b068ca99b', 'request_id': 'req-8bd40d05-4ddd-4034-86b4-82ec275e6025'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.268087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0858fbb3c6e47829e64e0fb96029977'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.268404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cd063f708ac4673b46ad20b068ca99b', 'request_id': 'req-8bd40d05-4ddd-4034-86b4-82ec275e6025'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.280200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0858fbb3c6e47829e64e0fb96029977'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.280949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0858fbb3c6e47829e64e0fb96029977'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.355410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cd063f708ac4673b46ad20b068ca99b', 'request_id': 'req-8bd40d05-4ddd-4034-86b4-82ec275e6025'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.370185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] The token cache is being invalidate because project 3cd063f708ac4673b46ad20b068ca99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:42.370591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.371214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.372780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 893/1812] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:42 2022] DELETE /identity/v3/projects/3cd063f708ac4673b46ad20b068ca99b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:42.377301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:42.378685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:42.418272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0858fbb3c6e47829e64e0fb96029977'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.427352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:42.428733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 920/1813] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:41 2022] DELETE /identity/v3/users/e0858fbb3c6e47829e64e0fb96029977 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:42.428733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:42.429928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:42.463012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd40d05-4ddd-4034-86b4-82ec275e6025 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:42.523215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4be74a-d022-4448-9d51-6bc58c641fab admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:42.523866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:42.524371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:42.525146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:42.525146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] PATH_INFO: `/v3/users/e81c0391ae3445828de1b83b5ce1bd0e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:42.526227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81c0391ae3445828de1b83b5ce1bd0e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:42.528490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:42.558464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:42.559596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:42.560265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:42.560265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] PATH_INFO: `/v3/users/91d2cc1a6f044e1aa3923bfd96cd8bad` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:42.561796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d2cc1a6f044e1aa3923bfd96cd8bad)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:42.563847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:42.750309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81c0391ae3445828de1b83b5ce1bd0e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.762299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81c0391ae3445828de1b83b5ce1bd0e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.781337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81c0391ae3445828de1b83b5ce1bd0e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.782159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81c0391ae3445828de1b83b5ce1bd0e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.799569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d2cc1a6f044e1aa3923bfd96cd8bad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.802108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d2cc1a6f044e1aa3923bfd96cd8bad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.849638 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d2cc1a6f044e1aa3923bfd96cd8bad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.863176 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81c0391ae3445828de1b83b5ce1bd0e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.867911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d2cc1a6f044e1aa3923bfd96cd8bad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.871167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 921/1814] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:42 2022] DELETE /identity/v3/users/e81c0391ae3445828de1b83b5ce1bd0e => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:42.955429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d2cc1a6f044e1aa3923bfd96cd8bad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:42.966410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 894/1815] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:42 2022] DELETE /identity/v3/users/91d2cc1a6f044e1aa3923bfd96cd8bad => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:42.995589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:42.997768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:43.023865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:43.116178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b631738-215d-42f0-b571-111d377f8783 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:43.211226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:43.212838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:43.213406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:43.213790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] PATH_INFO: `/v3/projects/eb52c0c496134b448b8023161f2d804b` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:43.215250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb52c0c496134b448b8023161f2d804b)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:43.239073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:43.369641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:43.371232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:43.393468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e871c9-a994-4d9b-ac6e-6620680584b4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:43.400468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:43.401129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:43.401408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:43.401683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] PATH_INFO: `/v3/projects/7231e0fd898e4ad39fa9f7de5a66ea5e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:43.403086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7231e0fd898e4ad39fa9f7de5a66ea5e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:43.405073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:43.445065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb52c0c496134b448b8023161f2d804b', 'request_id': 'req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.455648 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb52c0c496134b448b8023161f2d804b', 'request_id': 'req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.528216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb52c0c496134b448b8023161f2d804b', 'request_id': 'req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.531930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7231e0fd898e4ad39fa9f7de5a66ea5e', 'request_id': 'req-03f78d5f-97dc-4884-bc24-56a904ab1186'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.550992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] The token cache is being invalidate because project eb52c0c496134b448b8023161f2d804b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:43.551562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.552447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.553172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:43.557870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 922/1816] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:42 2022] DELETE /identity/v3/projects/eb52c0c496134b448b8023161f2d804b => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:43.557870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:43.557870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:43.561972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7231e0fd898e4ad39fa9f7de5a66ea5e', 'request_id': 'req-03f78d5f-97dc-4884-bc24-56a904ab1186'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.664098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b332-48c8-4b8b-9aea-3a3ceffb1a80 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:43.664480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7231e0fd898e4ad39fa9f7de5a66ea5e', 'request_id': 'req-03f78d5f-97dc-4884-bc24-56a904ab1186'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.684594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] The token cache is being invalidate because project 7231e0fd898e4ad39fa9f7de5a66ea5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:43.687998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.689917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-03f78d5f-97dc-4884-bc24-56a904ab1186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.695402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 895/1817] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:43 2022] DELETE /identity/v3/projects/7231e0fd898e4ad39fa9f7de5a66ea5e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:43.699838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:43.700204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:43.705385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:43.731371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:43.732729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:43.796938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:43.797613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:43.797896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:43.798175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] PATH_INFO: `/v3/projects/975eef09277c47769b75c01d5ab99b21` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:43.800479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=975eef09277c47769b75c01d5ab99b21)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:43.802440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:43.826688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 896/1818] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:43.829038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:43.832235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:43.846078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b04fe1-299d-4770-99cc-c43fd76ac792 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:43.852743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:43.853714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:43.854108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:43.854487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] PATH_INFO: `/v3/users/04d43b47c7064ba09f0f3d98eac0574b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:43.857036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d43b47c7064ba09f0f3d98eac0574b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:43.859246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:43.955819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '975eef09277c47769b75c01d5ab99b21', 'request_id': 'req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.962201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '975eef09277c47769b75c01d5ab99b21', 'request_id': 'req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:43.997226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:44.022937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '975eef09277c47769b75c01d5ab99b21', 'request_id': 'req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.044415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] The token cache is being invalidate because project 975eef09277c47769b75c01d5ab99b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:44.045015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.045730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f2f22ed1-5cae-4ed3-b6f7-fdbb9b3a3881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.047040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 923/1819] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:43 2022] DELETE /identity/v3/projects/975eef09277c47769b75c01d5ab99b21 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:44.049617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.049617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.049617 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.076377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:44.077999 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:44.093895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d43b47c7064ba09f0f3d98eac0574b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.095152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d43b47c7064ba09f0f3d98eac0574b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.127586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d43b47c7064ba09f0f3d98eac0574b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.170527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d43b47c7064ba09f0f3d98eac0574b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.261187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 924/1820] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:44.265545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.282470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd519ea-b1cd-4ff4-9d83-e8123de2a512 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.288628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.289253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.289539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.289816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.292124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d43b47c7064ba09f0f3d98eac0574b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.292283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:44.292600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.294389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.302745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 897/1821] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:43 2022] DELETE /identity/v3/users/04d43b47c7064ba09f0f3d98eac0574b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:44.307153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 925/1822] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:44.312338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.375829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfe36e-00d1-4268-ac45-5cc32ae13e7d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.381204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.381821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.382108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.384793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.384793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.385173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.418109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:44.474302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 898/1823] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:44 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:44.475927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.486202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7e35c-c83f-4d94-a911-1a96a26727ff admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.548336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.549034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.549321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.549603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.550250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.552240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.607495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:44.649018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 926/1824] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:44 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:44.653165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.670948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4154-496d-4c07-b9e3-cca27f7e9484 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.677597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.678218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.679504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.679504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.679504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.681149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.693031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 899/1825] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:44.697308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.707458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25f98-dec2-4942-b48b-599d0fe56ee1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.712156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.712745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.713022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.713290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.713964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.715580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.726039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 927/1826] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:44.730260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.740884 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c92faa-3b51-438f-b28b-c3dab5ccd6b9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.745764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.746948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.746948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.747642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] PATH_INFO: `/v3/projects/4cbb52e147ed4ee7bc16068e979aee7d/users/5b399ee3c11a4431800bddac3bc2c3bf/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.764373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cbb52e147ed4ee7bc16068e979aee7d, user_id=5b399ee3c11a4431800bddac3bc2c3bf, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.767060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b449f-aa0f-4726-ab61-73f0c39b3384 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.783241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:44.783241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:44.799082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d711f-6e29-429c-8825-08f1b31df5d5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:44.804311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:44.805001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.805277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.805554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] PATH_INFO: `/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.806689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00056f9bb4634b4d880254f0f9e6bfaf)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:44.808595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:44.868893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 900/1827] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:44 2022] PUT /identity/v3/projects/4cbb52e147ed4ee7bc16068e979aee7d/users/5b399ee3c11a4431800bddac3bc2c3bf/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:44.880810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:44.881324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:44.881803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:44.912716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] MFA Rules not processed for user `5b399ee3c11a4431800bddac3bc2c3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:44.912716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:44.963640 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00056f9bb4634b4d880254f0f9e6bfaf', 'request_id': 'req-822ec205-79c0-4bde-917d-4c50dcbc1298'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.973492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00056f9bb4634b4d880254f0f9e6bfaf', 'request_id': 'req-822ec205-79c0-4bde-917d-4c50dcbc1298'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:44.994303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 901/1828] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:44.999625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.058323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00056f9bb4634b4d880254f0f9e6bfaf', 'request_id': 'req-822ec205-79c0-4bde-917d-4c50dcbc1298'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:45.064905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b66c-d118-466b-ad1b-2aeadf5e8b7f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.073200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] The token cache is being invalidate because project 00056f9bb4634b4d880254f0f9e6bfaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:45.073549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:45.074181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-822ec205-79c0-4bde-917d-4c50dcbc1298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:45.075924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 928/1829] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:44 2022] DELETE /identity/v3/projects/00056f9bb4634b4d880254f0f9e6bfaf => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:45.082167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bab17e-1d5b-45f8-9a84-94f1ad41180d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.082460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bab17e-1d5b-45f8-9a84-94f1ad41180d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.082812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bab17e-1d5b-45f8-9a84-94f1ad41180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.103962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-46bab17e-1d5b-45f8-9a84-94f1ad41180d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:45.105170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-46bab17e-1d5b-45f8-9a84-94f1ad41180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:45.124057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:45.152364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.153269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.153677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.154011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.154799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.157222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.210069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 929/1830] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:45.221945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.225167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.225167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.259511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:45.261384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:45.278556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 902/1831] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:44 2022] POST /identity/v3/projects => generated 369 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:45.283108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.285077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 930/1832] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:45.286046 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:45.288724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.321939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a0d7b-c6de-4341-9f73-325fd742daff None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.339459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:45.405321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14900a-5970-4a70-a487-112e61b4e0d2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.416243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.417189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.417748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.418257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.419068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.421668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.458059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.458886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.459266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.460766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.462235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:45.463015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.465454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.479036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 931/1833] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:36:45.483150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.496279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae758a5f-07fe-4fb3-9979-548ae09a2cfa admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.502541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.503571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.504293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.504293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.505009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.507053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.555996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 903/1834] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:45.559826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.572465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcaeb6-db5a-43ae-9e9a-d2c2280f6ef2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.577878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.578859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.579288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.580420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.581249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.582936 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.598003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 904/1835] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:45.599326 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.610659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bd174-89da-4e75-b05d-1da87faefb7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.615575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.616236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.616527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.616803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] PATH_INFO: `/v3/projects/3edcb0f0b4f94794a595ec6047ff0622/users/cdfc332e0ce541f49bd8c221a11f6463/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.641004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 932/1836] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:45.641155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edcb0f0b4f94794a595ec6047ff0622, user_id=cdfc332e0ce541f49bd8c221a11f6463, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.641155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.644970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.728957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcca15-0916-4890-9b7b-d01e7f6a599d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.736573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.738904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.738904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.738904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.738904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.740496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.749610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 905/1837] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:45 2022] PUT /identity/v3/projects/3edcb0f0b4f94794a595ec6047ff0622/users/cdfc332e0ce541f49bd8c221a11f6463/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:45.758020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.838077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38fde-454a-464d-9b28-757a2c53eb2e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.841349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 933/1838] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:45.843360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.843970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.844522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.844800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.845480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.845912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.847204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.857397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2f177-c122-47f0-826c-35696c563aec admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.858407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 906/1839] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:45.862586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.868921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.870407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.870407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.870917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.871675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.873712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.886205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb26c-a008-4486-bf6e-76ddbeae7968 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.891166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 934/1840] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:45.892891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.895302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.896213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.896967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.897580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] PATH_INFO: `/v3/projects/3edcb0f0b4f94794a595ec6047ff0622/users/cdfc332e0ce541f49bd8c221a11f6463/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.899737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edcb0f0b4f94794a595ec6047ff0622, user_id=cdfc332e0ce541f49bd8c221a11f6463, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.902437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfbd1b-0e08-4011-86bb-2cabec6f80f1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.903834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2c22f-58bd-4a86-a60c-b5b34d178a7d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.909344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:45.909966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.910255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.910699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.911418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:45.913006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:45.928254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 935/1841] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:45.929133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:45.932352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 907/1842] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:45 2022] PUT /identity/v3/projects/3edcb0f0b4f94794a595ec6047ff0622/users/cdfc332e0ce541f49bd8c221a11f6463/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:45.940126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402ec24-0361-4b9e-a26c-cba6aaedd316 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:45.942807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744f48-39cf-47de-85ac-a42019f9c9f6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:45.943277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744f48-39cf-47de-85ac-a42019f9c9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:45.943707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744f48-39cf-47de-85ac-a42019f9c9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:45.964298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:45.983324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc744f48-39cf-47de-85ac-a42019f9c9f6 None None] MFA Rules not processed for user `cdfc332e0ce541f49bd8c221a11f6463`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:45.986695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc744f48-39cf-47de-85ac-a42019f9c9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:45.997475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:46.019088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.020808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.022270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.023433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] PATH_INFO: `/v3/projects/3df9b05f75e94bcb9d9a16975f70ca97/users/2e2cb54b72f94223b060711c1fb6828b/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.049090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df9b05f75e94bcb9d9a16975f70ca97, user_id=2e2cb54b72f94223b060711c1fb6828b, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.052165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e3a7e-a587-494e-9c37-671fba94aa14 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.090010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 908/1843] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:46.095733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.096196 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.096671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.098755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 909/1844] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 13:36:46.101627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:46.103391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.155322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 936/1845] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:45 2022] PUT /identity/v3/projects/3df9b05f75e94bcb9d9a16975f70ca97/users/2e2cb54b72f94223b060711c1fb6828b/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:46.156633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b338d99-a21f-4337-9ff5-a9972d343931 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.156931 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b338d99-a21f-4337-9ff5-a9972d343931 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.157209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b338d99-a21f-4337-9ff5-a9972d343931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.166348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcc438-474f-42f6-b26f-6e68c2f49919 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:46.182796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9b338d99-a21f-4337-9ff5-a9972d343931 None None] MFA Rules not processed for user `99786bcc97fa4b7eb9141fb3d509c28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:46.183942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9b338d99-a21f-4337-9ff5-a9972d343931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.219131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be828-4155-4844-970f-07742879207b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.219842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be828-4155-4844-970f-07742879207b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.220257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be828-4155-4844-970f-07742879207b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.220596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be828-4155-4844-970f-07742879207b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.248933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 937/1846] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:36:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:46.250592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.251026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.252205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.289610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be828-4155-4844-970f-07742879207b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.291100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be828-4155-4844-970f-07742879207b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.293529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] MFA Rules not processed for user `2e2cb54b72f94223b060711c1fb6828b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:46.295100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:46.295651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.302124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 910/1847] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:46.307418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be828-4155-4844-970f-07742879207b service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:46.308733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b64be828-4155-4844-970f-07742879207b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.323630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be828-4155-4844-970f-07742879207b service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:46.328858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.332096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.332096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.332096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.338448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.340057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.350037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 911/1848] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:46.351372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:46.352996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.365668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 938/1849] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:46.378460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:46.380417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.394538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db436-bd58-468e-9577-27c55aa2dc6c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:46.399144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.399850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.400200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.400539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.410072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.411635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.420751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 939/1850] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:46.421578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:46.430162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b295-3841-4a19-9eb4-13fd3a7f1096 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:46.434312 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.434896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.435159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.435433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.438084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:46.438555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74019ee9-e327-4dfb-be13-323815690bfe service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:46.444228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:46.446501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:46.446846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:46.447127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] PATH_INFO: `/v3/users/8665d60741af4d96a3ceb8f59ff458bb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:46.448611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8665d60741af4d96a3ceb8f59ff458bb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.451549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.502181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:46.517791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:46.519504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:46.528917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 940/1851] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:46.643488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8665d60741af4d96a3ceb8f59ff458bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:46.644376 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8665d60741af4d96a3ceb8f59ff458bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:46.653087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8665d60741af4d96a3ceb8f59ff458bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:46.666893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8665d60741af4d96a3ceb8f59ff458bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:46.749166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-252d6deb-449b-4360-8516-6c9dbc26d891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8665d60741af4d96a3ceb8f59ff458bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:46.755921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 912/1852] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:46 2022] DELETE /identity/v3/users/8665d60741af4d96a3ceb8f59ff458bb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:47.025921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:47.037761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e43929-efa7-4822-80e5-3577eb2575cf service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:47.100150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:47.101955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.101955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.101955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] PATH_INFO: `/v3/projects/1a49e667714043b9a821ac5c0b8b3663` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.103105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a49e667714043b9a821ac5c0b8b3663)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:47.105002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:47.238531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a49e667714043b9a821ac5c0b8b3663', 'request_id': 'req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:47.243311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a49e667714043b9a821ac5c0b8b3663', 'request_id': 'req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:47.406496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a49e667714043b9a821ac5c0b8b3663', 'request_id': 'req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:47.419427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] The token cache is being invalidate because project 1a49e667714043b9a821ac5c0b8b3663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:47.419679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:47.420359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79b91b1b-e0fb-4f4a-9822-f5c3aa1f378d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:47.422333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 941/1853] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:47 2022] DELETE /identity/v3/projects/1a49e667714043b9a821ac5c0b8b3663 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:47.435280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.435697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.436061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.457700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:47.459906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:47.520087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 913/1854] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:47.527972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.528266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.528549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.547886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:47.549109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:47.557175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 942/1855] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:47.561769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:47.568381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:47.572655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba92c-2a4f-4cc6-9fdc-b44085a421cc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:47.577756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:47.578553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.578885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.579164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.580485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:47.580789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:47.582585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:47.594305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 914/1856] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:36:47.598642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:47.608017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a71aa3-ff0d-4b88-87c7-5aafedf8a305 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:47.612827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:47.613427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.613711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.613984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.614661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:47.616561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:47.734549 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 943/1857] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:47 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:47.738805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:47.749123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784ad02-441c-48c6-9491-69a8d6646bb4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:47.808479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:47.809482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.809760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.810031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.810750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:47.812644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:47.953040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 915/1858] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:47 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:47.954343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:47.965335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a319a68-6fb6-4281-a6f8-5393dd4551b1 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:47.970213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:47.971588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:47.971940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:47.972327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:47.973041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:47.974743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:47.985470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 944/1859] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:47.989632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.004958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608293-903d-4551-a73a-1c4af67a8e68 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.009212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.009862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.010301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.010609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.011244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.012843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48fd22-9d97-47d4-a4e7-af1954e5c751 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.024424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 916/1860] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:48.029150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.039646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6f192-a979-44d5-86fb-f86f47c0dbdf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.044377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.045219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.045504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.045778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] PATH_INFO: `/v3/projects/ec51c4d2533742909b2dc6e87d01ec74/users/9a40ba66617b4256aa59e45785976096/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.061802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec51c4d2533742909b2dc6e87d01ec74, user_id=9a40ba66617b4256aa59e45785976096, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.067487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.149429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 945/1861] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:48 2022] PUT /identity/v3/projects/ec51c4d2533742909b2dc6e87d01ec74/users/9a40ba66617b4256aa59e45785976096/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:48.161655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.162020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.162388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.193027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] MFA Rules not processed for user `9a40ba66617b4256aa59e45785976096`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:48.194230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:48.270485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 917/1862] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:48.275140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.288643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:48.339336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519405a-6171-4501-b22a-874f43d34697 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.346112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.346772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.347058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.347336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.348021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.349937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.488045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 946/1863] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:48 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:48.492112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.502747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea5386-dcf5-4d30-92a5-697ff09783eb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.560325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.567227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.567227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.567227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.567227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.567227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.571992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:48.652994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 918/1864] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:48 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:48.659362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.671289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36e82-c82f-47c4-a4f1-07d582a4ac21 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.676192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.676946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.677233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.677507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.678157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.679807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.691205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 947/1865] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:48.695653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.706820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff895c4-fa46-40d5-91dc-e2684bd870dc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.712515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.713378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.713811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.714209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] PATH_INFO: `/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb/users/791d1a0bfda24fc68adeee8a6d11ca5b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.730398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2aa99b8a3a4a80a2a578a6f3bc29cb, user_id=791d1a0bfda24fc68adeee8a6d11ca5b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.733203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.847122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 919/1866] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:48 2022] PUT /identity/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb/users/791d1a0bfda24fc68adeee8a6d11ca5b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:48.851227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.861290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667d74b-e006-4961-9cf1-dcf5d72eab83 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.928423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.929050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.929333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.929810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.930397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.931915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26556d-f23f-4b26-9067-b9cd2c2647c9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.943798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 948/1867] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:48.947872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:48.959630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc46f49-ec22-4082-aa8e-6679e7da5cad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:48.964392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:48.965050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.965329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.965603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] PATH_INFO: `/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb/users/791d1a0bfda24fc68adeee8a6d11ca5b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:48.967685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2aa99b8a3a4a80a2a578a6f3bc29cb, user_id=791d1a0bfda24fc68adeee8a6d11ca5b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:48.972281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:48.982033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 920/1868] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:48 2022] PUT /identity/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb/users/791d1a0bfda24fc68adeee8a6d11ca5b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:48.993825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:48.994268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:48.994632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:49.038631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] MFA Rules not processed for user `791d1a0bfda24fc68adeee8a6d11ca5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:49.040146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:49.110279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 949/1869] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:49.121573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:49.123372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:49.184603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2e9a8-1098-42f3-be7e-bdd1124bbd4b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:49.188422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:49.189624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:49.189905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:49.190182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:49.199648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:49.201082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:49.210521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 921/1870] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:49.461939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:49.476132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ffda3-a1f5-422e-baf7-8ccef88fc8a8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:49.479765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:49.480376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:49.480666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:49.480944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:49.485173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:49.541282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:49.542802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8fb93-5908-4a5f-bb5f-930bf7ac22ec service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:49.552434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 950/1871] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:51.237558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:51.240354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:51.339802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ffb80-d6ba-4317-a048-fad2ba8e0d03 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:51.344654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:51.346362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:51.346362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:51.346578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] PATH_INFO: `/v3/users/1f9be369a0e549aebdb15a310b4eec60` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:51.347721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9be369a0e549aebdb15a310b4eec60)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:51.349727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:51.569974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9be369a0e549aebdb15a310b4eec60'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:51.570760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9be369a0e549aebdb15a310b4eec60'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:51.578866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9be369a0e549aebdb15a310b4eec60'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:51.579985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:51.593609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9be369a0e549aebdb15a310b4eec60'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:51.708935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9be369a0e549aebdb15a310b4eec60'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:51.718895 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 922/1872] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:51 2022] DELETE /identity/v3/users/1f9be369a0e549aebdb15a310b4eec60 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:51.721553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:51.722379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:51.722703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:51.742477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:51.743870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:51.806750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 951/1873] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:51.820152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:51.822475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:51.894891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b9879-6e12-40b4-a1a6-e021f15a7aef admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:51.899043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:51.899666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:51.899948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:51.900222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:51.911715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:51.913192 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:51.933381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 923/1874] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:52.169775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.173325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.183526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f1fc9-ede4-42ae-9da6-198652162bcd None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.188782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.189466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.189750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.190026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] PATH_INFO: `/v3/users/5b399ee3c11a4431800bddac3bc2c3bf` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.191198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b399ee3c11a4431800bddac3bc2c3bf)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:52.193137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:52.373550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.375166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.390276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d3f0-0e59-4998-b6dd-7d2001356701 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.397582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.399551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.399551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.399551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] PATH_INFO: `/v3/projects/f739f9e94ae145c887bbb3ecf8919b19` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.400501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f739f9e94ae145c887bbb3ecf8919b19)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:52.402594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:52.462446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b399ee3c11a4431800bddac3bc2c3bf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.471475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b399ee3c11a4431800bddac3bc2c3bf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.486360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b399ee3c11a4431800bddac3bc2c3bf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.487118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b399ee3c11a4431800bddac3bc2c3bf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.538723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f739f9e94ae145c887bbb3ecf8919b19', 'request_id': 'req-f2261205-a88c-420c-9d52-208c26a58f15'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.546105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f739f9e94ae145c887bbb3ecf8919b19', 'request_id': 'req-f2261205-a88c-420c-9d52-208c26a58f15'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.583416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-c09a1147-dacc-42c0-9d5a-dabece8ae2ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b399ee3c11a4431800bddac3bc2c3bf'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.590622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 952/1875] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:52 2022] DELETE /identity/v3/users/5b399ee3c11a4431800bddac3bc2c3bf => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:52.595150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.595443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.596494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.618293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:52.619463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.631378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:52.637200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f739f9e94ae145c887bbb3ecf8919b19', 'request_id': 'req-f2261205-a88c-420c-9d52-208c26a58f15'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.652104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] The token cache is being invalidate because project f739f9e94ae145c887bbb3ecf8919b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:52.652534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.653424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f2261205-a88c-420c-9d52-208c26a58f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:52.659753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 924/1876] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:52 2022] DELETE /identity/v3/projects/f739f9e94ae145c887bbb3ecf8919b19 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:52.660620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec842827-5404-4a8f-8652-d38a6c039f1c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.660909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec842827-5404-4a8f-8652-d38a6c039f1c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.661190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec842827-5404-4a8f-8652-d38a6c039f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.690154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ec842827-5404-4a8f-8652-d38a6c039f1c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:52.690343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 953/1877] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:52.690427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec842827-5404-4a8f-8652-d38a6c039f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.698549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 925/1878] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:52.700087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.702108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.709953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.709953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.709953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.727872 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:52.729296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:52.743545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 926/1879] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:52.744485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.756898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3491df-e8ed-47d7-912c-b24cf6b958cb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.763118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.764017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.764442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.764796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.765543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:52.765929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:52.767780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:52.770969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cc9f1-1c44-4216-8938-c648d2f36cdd None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.775018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.775647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.775936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.776453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.782304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 927/1880] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:36:52.787969 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.788151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:52.789495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:52.800551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044df49-b359-48a8-82a4-478b8044ca94 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.804722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 954/1881] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:52.807779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.808411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.809163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:52.809470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:52.810108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:52.812663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:52.916698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 928/1882] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:52 2022] POST /identity/v3/projects => generated 359 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:52.922478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:52.933416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7b85e-c520-4360-bd1d-a570d5f77277 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:52.997415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:52.998035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:52.998317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.002637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.002637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.002637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.105907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 955/1883] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:52 2022] POST /identity/v3/users => generated 280 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:53.119101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.141679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c408a6-57a0-4075-b906-c7efc745727c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.154915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.156145 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.156431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.157071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.157372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.159094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.172771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:53.178672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 929/1884] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:53.182958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.195967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7690904-d04c-4d1b-8766-1dcff5e7844d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.201243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.201980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.202370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.202799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.203651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.205478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.218035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 956/1885] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:53.222558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.232681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe0904-02d1-4465-8c90-1d50915300bc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.238037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.238802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.239079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.239356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] PATH_INFO: `/v3/projects/4e30ead23f4b40a5a497f86c8da5ca0d/users/d5065f2d72074aaa972c1a4ecae4b47e/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.255159 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e30ead23f4b40a5a497f86c8da5ca0d, user_id=d5065f2d72074aaa972c1a4ecae4b47e, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.257886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1351be-6a87-401d-8463-3edf97212720 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.322418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.324660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.348243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d718c-6d24-46bf-9df4-10211b4a3da0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.353199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.353965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.354249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.354554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] PATH_INFO: `/v3/projects/4cbb52e147ed4ee7bc16068e979aee7d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.355670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cbb52e147ed4ee7bc16068e979aee7d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.357607 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.367749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 930/1886] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:53 2022] PUT /identity/v3/projects/4e30ead23f4b40a5a497f86c8da5ca0d/users/d5065f2d72074aaa972c1a4ecae4b47e/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:53.379227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.379856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.379856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.417516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] MFA Rules not processed for user `d5065f2d72074aaa972c1a4ecae4b47e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:53.418941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.452392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:53.462256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cbb52e147ed4ee7bc16068e979aee7d', 'request_id': 'req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.468020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cbb52e147ed4ee7bc16068e979aee7d', 'request_id': 'req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.490402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 931/1887] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:53.526102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cbb52e147ed4ee7bc16068e979aee7d', 'request_id': 'req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.528154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.529767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.542384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] The token cache is being invalidate because project 4cbb52e147ed4ee7bc16068e979aee7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:53.542980 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.543847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.545842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 957/1888] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:53 2022] DELETE /identity/v3/projects/4cbb52e147ed4ee7bc16068e979aee7d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:53.550256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.551660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.592171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069d76c-4a2d-42c0-9db5-f00e9a760113 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.616644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1d1cc-e0c9-4841-a1ce-e5cb1db28cf8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.621406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.622132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.622411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.622715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] PATH_INFO: `/v3/users/cdfc332e0ce541f49bd8c221a11f6463` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.623893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfc332e0ce541f49bd8c221a11f6463)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.625823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.645838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.646452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.646784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.647062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.648886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.710242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.711763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.721544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 932/1889] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:53.739659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:53.741116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:53.756690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4723-b658-4bf9-a915-1bd82ea8f805 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:53.761481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:53.762236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:53.762579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:53.762944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] PATH_INFO: `/v3/users/d5065f2d72074aaa972c1a4ecae4b47e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:53.764114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5065f2d72074aaa972c1a4ecae4b47e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:53.766096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:53.789410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:53.803031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfc332e0ce541f49bd8c221a11f6463'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.812363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfc332e0ce541f49bd8c221a11f6463'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.826858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfc332e0ce541f49bd8c221a11f6463'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.827660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfc332e0ce541f49bd8c221a11f6463'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.931608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-318fd2aa-f8a7-49ee-a948-ed8312246996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfc332e0ce541f49bd8c221a11f6463'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.938278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 958/1890] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:53 2022] DELETE /identity/v3/users/cdfc332e0ce541f49bd8c221a11f6463 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:53.965054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5065f2d72074aaa972c1a4ecae4b47e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.965870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5065f2d72074aaa972c1a4ecae4b47e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.975235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5065f2d72074aaa972c1a4ecae4b47e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:53.993746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5065f2d72074aaa972c1a4ecae4b47e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.108992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5065f2d72074aaa972c1a4ecae4b47e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.115436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 933/1891] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:53 2022] DELETE /identity/v3/users/d5065f2d72074aaa972c1a4ecae4b47e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:54.120070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.120357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.120629 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.140553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:54.141842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.192143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:54.217274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 959/1892] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:54.227905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:54.230085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.299836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac284-c7b4-434f-a639-68c8cd9d790c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:54.303724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:54.304416 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.304766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.305115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.317096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:54.318560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:54.328220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 934/1893] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:54.420795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:54.422966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.441407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c20f88-1635-442c-91e9-68f44494eee8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:54.450013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:54.450754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.451039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.451317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] PATH_INFO: `/v3/projects/3edcb0f0b4f94794a595ec6047ff0622` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.452644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edcb0f0b4f94794a595ec6047ff0622)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:54.454736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:54.598746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edcb0f0b4f94794a595ec6047ff0622', 'request_id': 'req-d3b7265c-e772-471b-974e-2a86b7a150a1'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.600346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:54.603248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.604905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edcb0f0b4f94794a595ec6047ff0622', 'request_id': 'req-d3b7265c-e772-471b-974e-2a86b7a150a1'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.623053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1749be-ea75-405e-8147-78ed4068044e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:54.676140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:54.686260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:54.686948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.687229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.687534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] PATH_INFO: `/v3/users/5c57cd45b190493faa03afef97356b82` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.688660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c57cd45b190493faa03afef97356b82)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:54.691197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:54.744043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edcb0f0b4f94794a595ec6047ff0622', 'request_id': 'req-d3b7265c-e772-471b-974e-2a86b7a150a1'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.761571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] The token cache is being invalidate because project 3edcb0f0b4f94794a595ec6047ff0622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:54.761955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.762706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3b7265c-e772-471b-974e-2a86b7a150a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.765393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 960/1894] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:54 2022] DELETE /identity/v3/projects/3edcb0f0b4f94794a595ec6047ff0622 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:54.773537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a8408-ec34-45c0-a6d8-7aae32cd55bf None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.774785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a8408-ec34-45c0-a6d8-7aae32cd55bf None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.774785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a8408-ec34-45c0-a6d8-7aae32cd55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.794767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-291a8408-ec34-45c0-a6d8-7aae32cd55bf None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:54.795985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-291a8408-ec34-45c0-a6d8-7aae32cd55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.804533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 961/1895] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:54.812502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.812814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.813093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.834281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:54.835524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.843641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 962/1896] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:54.853912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:54.881045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e6403-087a-415e-814e-9a6ac6931496 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:54.884510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c57cd45b190493faa03afef97356b82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.887014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c57cd45b190493faa03afef97356b82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.889243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:54.891057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.891336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.891611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.892280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:54.894186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:54.896700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:54.905141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c57cd45b190493faa03afef97356b82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.915604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 963/1897] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:54.916472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:54.918210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:54.922748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c57cd45b190493faa03afef97356b82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:54.931158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93473f-f17d-4147-82f7-84a16845c556 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:54.937430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:54.940115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:54.940115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:54.940115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] PATH_INFO: `/v3/projects/4e30ead23f4b40a5a497f86c8da5ca0d` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:54.940574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e30ead23f4b40a5a497f86c8da5ca0d)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:54.943799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.002681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c57cd45b190493faa03afef97356b82'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.013216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 935/1898] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:36:54 2022] DELETE /identity/v3/users/5c57cd45b190493faa03afef97356b82 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:55.013216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.013216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.080739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ec6a8-7e7e-449e-a932-11aed84fd602 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.085528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.086149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.086462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.091566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.091566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.091566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9502a-5933-40c5-8c47-66bf968ba64a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.098612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e30ead23f4b40a5a497f86c8da5ca0d', 'request_id': 'req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.104520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e30ead23f4b40a5a497f86c8da5ca0d', 'request_id': 'req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.191419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 936/1899] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:55.192708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eec73-0933-4758-a9a3-1a39ddf14ff8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.193047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eec73-0933-4758-a9a3-1a39ddf14ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.193309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eec73-0933-4758-a9a3-1a39ddf14ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.212494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ce6eec73-0933-4758-a9a3-1a39ddf14ff8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:55.213664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ce6eec73-0933-4758-a9a3-1a39ddf14ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.217826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e30ead23f4b40a5a497f86c8da5ca0d', 'request_id': 'req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.230337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] The token cache is being invalidate because project 4e30ead23f4b40a5a497f86c8da5ca0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:55.232578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.232578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:55.233110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 964/1900] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:54 2022] DELETE /identity/v3/projects/4e30ead23f4b40a5a497f86c8da5ca0d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:55.233983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.235378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.263422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:55.277230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 937/1901] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:55.278568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.279049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.279373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.299738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:55.301068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.303130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8ffc5-e65f-45fb-9f24-72efd1fe1611 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.308010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.308627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.309187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.316467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.316591 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 938/1902] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:55.316687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.316773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.316858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.316943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.370790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42d69-9179-4b02-95d9-7927b8dcb616 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.374404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.375031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.375309 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.375579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.385522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.386974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8a3cc-8d20-4b8f-ad64-7c29f2e80126 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.396782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 939/1903] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:55.397951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.398334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.398906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.404009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 965/1904] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:55.406458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.418291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524dcb-5f1c-4cf7-8174-3d61f04f4b9c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.419869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:55.421110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.426083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.426875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.427260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.427933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.428422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.430289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.430663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 940/1905] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:55.434838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.446554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 966/1906] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:55.449917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111747f2-6a8f-42e9-820a-e59b0f2c31c2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.451534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.455348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.455973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.456264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.456539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.457219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:55.457513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.459236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.462314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edbbcf-74e0-4cb7-a6d7-f8f2e3db0913 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.476153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 941/1907] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:55.476339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.476339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.477048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.477712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.477998 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.478937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.480785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.488597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed54725-a635-46ee-93a7-53f7fa4c7e82 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.493427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 967/1908] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:55.493908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.494740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.495117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.495492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.496347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.497384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.498410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.510621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e094a-00bf-4909-89c0-5fdcf758a3b4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.519262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.519262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.519262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.519262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] PATH_INFO: `/v3/projects/c039bc44943143c68ab1c117ce00d8ff/users/be6701871c18479fa1694abd41b97695/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.534635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c039bc44943143c68ab1c117ce00d8ff, user_id=be6701871c18479fa1694abd41b97695, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.537403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0760d-8ac7-489c-ba82-af668ce11f65 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.537939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:55.595315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 968/1909] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:55 2022] PUT /identity/v3/projects/c039bc44943143c68ab1c117ce00d8ff/users/be6701871c18479fa1694abd41b97695/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:55.604890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 942/1910] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:55.607271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.607585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.607869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.608973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.641619 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] MFA Rules not processed for user `be6701871c18479fa1694abd41b97695`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:55.643149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.695396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256a64a-8e3b-4d1c-9e23-1055711245ed admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.702563 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.703185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.703608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.704052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.705802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.708795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.728909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 969/1911] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:55.733405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.744354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d062f8-52ff-4775-81a5-a488b35fb39e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.749379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.749976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.750263 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.750572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.751208 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.753113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.843029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 943/1912] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:55.845106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 970/1913] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:55.846066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.847253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.853646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:55.868492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789dfd4-6a45-448c-951f-d480197c402e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.915211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e4c79-6c41-4b31-92f9-640c17f0b0c5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.925564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.926196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.926480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.927170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.927438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.929137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.940256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 971/1914] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:55.941042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:55.951511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.952157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.952284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da92b4-b617-4de7-9c90-31679031f3f4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:55.952472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.952712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] PATH_INFO: `/v3/projects/304d441864be4cc29cb830edbd9a5af3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.953833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=304d441864be4cc29cb830edbd9a5af3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.956506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.959050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:55.959050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:55.959402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:55.959661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:55.960375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:55.962454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:55.977737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:56.069524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 972/1915] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:55 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:56.070359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.083820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304d441864be4cc29cb830edbd9a5af3', 'request_id': 'req-e2607ec9-d533-46e5-bb94-d0b4c86da49d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:56.090250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da889-a6b2-4c04-ab1a-383c5de1a5a4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.091482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304d441864be4cc29cb830edbd9a5af3', 'request_id': 'req-e2607ec9-d533-46e5-bb94-d0b4c86da49d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:56.153211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.153211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.153521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.153686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.154336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.156031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.166326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 973/1916] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:56.167187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.179577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525f870-b4dc-491e-95ae-bb4da6d8f7af admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.183435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304d441864be4cc29cb830edbd9a5af3', 'request_id': 'req-e2607ec9-d533-46e5-bb94-d0b4c86da49d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:56.185059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.185822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.186106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.187440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.192667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.192667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffa30d-4d9a-4beb-994a-c4a99fd5f9e7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.197493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] The token cache is being invalidate because project 304d441864be4cc29cb830edbd9a5af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:36:56.197901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:56.198721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:36:56.200322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 944/1917] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:36:55 2022] DELETE /identity/v3/projects/304d441864be4cc29cb830edbd9a5af3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:56.201161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.202725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 974/1918] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:56.202852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.205139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.205599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.205884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.222286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:56.222286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607ec9-d533-46e5-bb94-d0b4c86da49d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.227527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.228502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:56.228629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.229671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.229791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.230275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] PATH_INFO: `/v3/projects/b8a5c58197a54720b56d45208b83183a/users/feda2cafbc9843d586895079e7a55a2c/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.238737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 975/1919] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:56.239986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.241362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.250256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a5c58197a54720b56d45208b83183a, user_id=feda2cafbc9843d586895079e7a55a2c, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.251278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.255292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf120f9e-1a32-4b72-b7c0-ad531d300f3c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.260359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.261011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.261293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.261566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.262215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.263841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd2b76-2edb-4d27-9ec7-1ac62a20638e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.275347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 976/1920] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:56.278606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.278606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.278606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.295628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:56.296819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.306195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 977/1921] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:56.313076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.318112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1498e-6e4f-4cb7-8fc4-ac0e20f2f59c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.322890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.323519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.323801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.324078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] PATH_INFO: `/v3/projects/e11e7adfd7fe4f389171a99a90a14f08/users/374628a847c14152b0349e5781f4907e/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.338762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 945/1922] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:56 2022] PUT /identity/v3/projects/b8a5c58197a54720b56d45208b83183a/users/feda2cafbc9843d586895079e7a55a2c/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:56.339743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.339956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11e7adfd7fe4f389171a99a90a14f08, user_id=374628a847c14152b0349e5781f4907e, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.342662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18fdc-9d5f-4eda-abf9-dcf7111c2a44 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.400173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 978/1923] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:56 2022] PUT /identity/v3/projects/e11e7adfd7fe4f389171a99a90a14f08/users/374628a847c14152b0349e5781f4907e/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:36:56.401510 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.401636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2c2c-e1a8-4a6c-887e-3bc6a6795fe5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.401802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.403213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.433013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] MFA Rules not processed for user `feda2cafbc9843d586895079e7a55a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:56.436450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.468303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.469048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.469330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.469602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.470247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:36:56.471077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.472666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067b642-9ff0-41f5-8e5e-c2485cb3eb2e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.489007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 946/1924] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:36:56.489007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.489007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.489007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.494653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:56.499569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:56.520216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] MFA Rules not processed for user `374628a847c14152b0349e5781f4907e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:56.521473 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.543115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 979/1925] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:56.544195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.559956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b007a-d28d-4f27-9e3d-747cad8ee96b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.567347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.568463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.568463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.568928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.569795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.571822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.591797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 947/1926] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:56.592821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.596943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:56.652918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9d622-f279-46c3-9346-dd5a324d401b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.656626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.657223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.657501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.657770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.666570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.670904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.677020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 948/1927] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:56.677794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.689135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a3a8c-7145-4b85-b6e1-0d7b2ff32c12 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.700099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 980/1928] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:56.701959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.702894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.703189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.703482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.704127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.704340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.706793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.771974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc272f-90bc-4301-b6d2-005b53f59b66 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.776830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.777457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.777754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.778027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.779412 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.785832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.814540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 949/1929] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:56.817521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.880120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98088-5aaf-47fd-8908-3c5cdba1c529 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.884777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.885380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.885662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.885935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.886613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.888540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.901522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 981/1930] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:36:56.902654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.904434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:56.913221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fce08a-83fb-4945-8712-b31ad9f88173 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.918893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.919485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.919875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.920153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.920788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.922367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.932290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 982/1931] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:56.937530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.946692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e057fbd-e33e-4af1-a2ae-323fb90967b0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.951211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.951754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.952032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.952300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.952908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:56.954413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:56.965136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 983/1932] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:56.968870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.978983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eb7ab-65ec-488b-b9c2-55066588cf1c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:56.983844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:56.984059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 950/1933] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:36:56 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:36:56.984480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:56.984758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:56.985032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] PATH_INFO: `/v3/projects/62fb03abd1f64d748b132fa450f641f8/users/b69eb168c92343b5819d4795eb85dd7a/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:56.988037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:56.997812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280914f7-fa34-4962-adf1-11a9d18b69c6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.003893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62fb03abd1f64d748b132fa450f641f8, user_id=b69eb168c92343b5819d4795eb85dd7a, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.006195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.007020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.007143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.007243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10006157-f1b1-4dd8-96a4-74edfefceef7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.007411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.008062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.009678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.021849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 951/1934] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:36:57.025908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:57.036660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea3653-5184-4238-b0c6-4c2b6b6dd24d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.041497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.042282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.042593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.042870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] PATH_INFO: `/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae/users/7e2d05a3e43247d1bdd9c86f3cbf3518/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.058336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae2e1440fd4453e95706bc0c7f8efae, user_id=7e2d05a3e43247d1bdd9c86f3cbf3518, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.061274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.103615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 984/1935] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:56 2022] PUT /identity/v3/projects/62fb03abd1f64d748b132fa450f641f8/users/b69eb168c92343b5819d4795eb85dd7a/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:57.109302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.109587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.109863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.121525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 952/1936] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:57 2022] PUT /identity/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae/users/7e2d05a3e43247d1bdd9c86f3cbf3518/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:57.127462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:57.132869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:57.134088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:57.188503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:57.194336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472211a-0962-46d0-8271-dcd0087a9f66 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.201134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 985/1937] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:57.201273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.201273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.201273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.201273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.202964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.202964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.208281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:57.209758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:57.216809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 953/1938] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:36:57.217796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:57.227482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b2a05-b0e2-417f-8de5-47983217ef59 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.232418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.233119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.233438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.233720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] PATH_INFO: `/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae/users/7e2d05a3e43247d1bdd9c86f3cbf3518/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.235711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae2e1440fd4453e95706bc0c7f8efae, user_id=7e2d05a3e43247d1bdd9c86f3cbf3518, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.238397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b33b73-68af-4ebd-8621-80316e33320e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.249853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 954/1939] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:36:57 2022] PUT /identity/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae/users/7e2d05a3e43247d1bdd9c86f3cbf3518/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:36:57.260629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.262332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.262332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.273408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffc41a-9354-44fc-b075-8cf7764bdaf1 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.307135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] MFA Rules not processed for user `7e2d05a3e43247d1bdd9c86f3cbf3518`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:57.310034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:57.356831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.357887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.359169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.359642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.409293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 955/1940] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:36:57.418627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:57.449755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:57.456171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.457718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.468063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 986/1941] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:57.495870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e5842-8ff1-4cd7-bed7-ad83fbe8930d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:57.500636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:57.500636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:57.500853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:57.500949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:57.510829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:57.512343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efc7a4-46c6-438a-9b57-76053fafc171 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:57.522590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 956/1942] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:36:58.610091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:58.628571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f342a-583c-469c-b31b-5a486231dc5c service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:58.646780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029cde-c4be-4cad-8516-0976712db8ab None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:58.647239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:58.647360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029cde-c4be-4cad-8516-0976712db8ab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:58.647360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029cde-c4be-4cad-8516-0976712db8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:58.647532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:58.647532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:58.647532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:58.647763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:58.692650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ec029cde-c4be-4cad-8516-0976712db8ab None None] MFA Rules not processed for user `b69eb168c92343b5819d4795eb85dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:36:58.694460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec029cde-c4be-4cad-8516-0976712db8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:58.728389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:58.730035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:58.740830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 987/1943] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:36:58.777017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 957/1944] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:36:58.785181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:36:58.786662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:36:58.794301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:36:58.798355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a931a3-c8aa-4afa-8ce3-4cba387ceb6c service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:36:58.809230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:36:58.809230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:36:58.809230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:36:58.809230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:36:58.812308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:36:58.813768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:36:58.823518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 988/1945] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:36:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:02.673775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906d29-cb42-4958-8a85-7eae114df761 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:02.674312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906d29-cb42-4958-8a85-7eae114df761 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:02.674410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906d29-cb42-4958-8a85-7eae114df761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:02.694773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eb906d29-cb42-4958-8a85-7eae114df761 None None] MFA Rules not processed for user `b69eb168c92343b5819d4795eb85dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:02.695970 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eb906d29-cb42-4958-8a85-7eae114df761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:02.703172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 958/1946] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:03.990024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:04.002569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac018-97dc-4d12-99cb-6812cd75bc2b service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:04.008825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:04.009654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:04.010284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:04.010606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] PATH_INFO: `/v3/users/feda2cafbc9843d586895079e7a55a2c` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:04.012145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feda2cafbc9843d586895079e7a55a2c)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:04.014073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:04.202384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feda2cafbc9843d586895079e7a55a2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:04.209326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feda2cafbc9843d586895079e7a55a2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:04.221467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feda2cafbc9843d586895079e7a55a2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:04.222238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feda2cafbc9843d586895079e7a55a2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:04.297349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feda2cafbc9843d586895079e7a55a2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:04.304030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 989/1947] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:37:03 2022] DELETE /identity/v3/users/feda2cafbc9843d586895079e7a55a2c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:04.309016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:04.309508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:04.309602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:04.328960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:04.330174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:04.389528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 959/1948] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:04.399267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:04.400703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:04.461382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79ac6c-fc55-4335-842e-f442d1e1c160 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:04.465095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:04.465728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:04.466009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:04.466281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:04.476027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:04.477452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35375fbc-4208-4261-ab27-12956b99ce6d service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:04.489982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 990/1949] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:04.880547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:04.882026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:04.897352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00d9c9-fee5-4593-8f99-9df17e83b06c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:04.902321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:04.903193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:04.903304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:04.903843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] PATH_INFO: `/v3/projects/b8a5c58197a54720b56d45208b83183a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:04.905597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a5c58197a54720b56d45208b83183a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:04.907582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.006661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:05.012817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a5c58197a54720b56d45208b83183a', 'request_id': 'req-bafdb53e-1903-47f0-8b13-ba494e6ab80a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:05.018708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a5c58197a54720b56d45208b83183a', 'request_id': 'req-bafdb53e-1903-47f0-8b13-ba494e6ab80a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:05.114068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a5c58197a54720b56d45208b83183a', 'request_id': 'req-bafdb53e-1903-47f0-8b13-ba494e6ab80a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:05.128036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] The token cache is being invalidate because project b8a5c58197a54720b56d45208b83183a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:37:05.128413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:05.129062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafdb53e-1903-47f0-8b13-ba494e6ab80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:05.130809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 960/1950] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:37:04 2022] DELETE /identity/v3/projects/b8a5c58197a54720b56d45208b83183a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:05.137146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.137638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.137837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.159912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:05.161696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:05.225233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 991/1951] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:05.233907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.234199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.238478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.255669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:05.256921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:05.264835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 961/1952] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:05.269007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.279261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98265fd4-3efa-493d-b194-9817f6256ac4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.283886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.284488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.284768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.285041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.285692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:37:05.286042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.287671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.289126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:05.299537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 992/1953] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:37:05.303818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.313527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ff57-4924-4c95-834b-fb438e3b57f8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.318931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.319531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.319932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.320223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.320861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.322791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.429753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 962/1954] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:37:05 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:37:05.434325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.444708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdadfa-79f9-401e-9b6b-cf7b18867e8c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.500951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.501571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.501840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.502116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.502775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.505236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.592015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 993/1955] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:37:05 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:37:05.593661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.604438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840f519-b3c2-4b86-971a-66dd395dfac8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.609043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.609639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.609954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.610410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.611087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.612663 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.623493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 963/1956] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:37:05.627602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.638144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ff2a4-ae34-4a25-bdb8-64891acedd8a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.642730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.643321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.643601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.643870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.644552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.646159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58e03-26a8-4c1a-8333-c3a7d49d79e2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.656469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 994/1957] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:37:05.660454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.670283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40658c99-188b-44eb-a26a-5881c3b7ec08 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.674898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.675544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.675827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.676101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] PATH_INFO: `/v3/projects/c20b3147c0d14452b9c5f22801c89990/users/eb7a8349031b4f87a4f477e4a4e6ba2b/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.691203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20b3147c0d14452b9c5f22801c89990, user_id=eb7a8349031b4f87a4f477e4a4e6ba2b, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.694498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.783377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 964/1958] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:37:05 2022] PUT /identity/v3/projects/c20b3147c0d14452b9c5f22801c89990/users/eb7a8349031b4f87a4f477e4a4e6ba2b/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:37:05.788196 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1f51-8eaf-4607-8073-f7b423e5f29f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.788485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1f51-8eaf-4607-8073-f7b423e5f29f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.796459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1f51-8eaf-4607-8073-f7b423e5f29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.809131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bedd1f51-8eaf-4607-8073-f7b423e5f29f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:05.810610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bedd1f51-8eaf-4607-8073-f7b423e5f29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:05.881275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 995/1959] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:05.891188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:05.893007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:05.950862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809631c-4c3e-4cc1-97c5-1cd80e8c69b9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:05.954696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:05.955331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:05.955610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:05.955883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:05.965401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:05.966857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:05.976808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 965/1960] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:07.211513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6d54-390b-45e1-93fa-94ee11163c17 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:07.212056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6d54-390b-45e1-93fa-94ee11163c17 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:07.212149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6d54-390b-45e1-93fa-94ee11163c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:07.241668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5c6c6d54-390b-45e1-93fa-94ee11163c17 None None] MFA Rules not processed for user `eb7a8349031b4f87a4f477e4a4e6ba2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:07.242386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c6c6d54-390b-45e1-93fa-94ee11163c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:07.315352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 996/1961] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:07.321251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:07.322754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:07.380741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff971e8-6016-4e25-8a50-d44a415f61f1 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:07.384301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:07.384902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:07.385185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:07.385462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:07.393564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:07.395544 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:07.404144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 966/1962] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:10.076897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:10.077392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:10.077702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:10.085826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:10.098829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] MFA Rules not processed for user `eb7a8349031b4f87a4f477e4a4e6ba2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:10.100132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:10.107053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 997/1963] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:12.180129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:12.195084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33145dc0-3e75-423f-862b-9d0560c5e268 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:12.199604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:12.200379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:12.200805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:12.201221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:12.204690 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:12.219989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:12.275135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:12.276672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474dea6-95f6-4819-aa4f-1f6ddec5bdf6 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:12.286074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 967/1964] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:16.457389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:16.458870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:16.475728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36cec-efa2-4c33-b729-c13db7b7e94f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:16.481722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:16.482726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:16.483222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:16.483703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] PATH_INFO: `/v3/users/be6701871c18479fa1694abd41b97695` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:16.485317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6701871c18479fa1694abd41b97695)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:16.487800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:16.674039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6701871c18479fa1694abd41b97695'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:16.682153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6701871c18479fa1694abd41b97695'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:16.696178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6701871c18479fa1694abd41b97695'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:16.697057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6701871c18479fa1694abd41b97695'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:16.771459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6701871c18479fa1694abd41b97695'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:16.778972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 998/1965] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:37:16 2022] DELETE /identity/v3/users/be6701871c18479fa1694abd41b97695 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:16.783430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:16.783942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:16.784042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:16.804064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:16.805279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:16.886392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 968/1966] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:16.899197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:16.900711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:16.981024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d15d9-c7ab-41e1-880c-fba817e18423 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:16.985537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:16.986279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:16.987029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:16.987319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:17.006762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:17.008221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:17.021459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 999/1967] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:18.124011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:18.125742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:18.138301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2e48-d359-4245-8a1f-211f1f7034f6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:18.143501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:18.144135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:18.144414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:18.147378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] PATH_INFO: `/v3/projects/c039bc44943143c68ab1c117ce00d8ff` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:18.147378 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c039bc44943143c68ab1c117ce00d8ff)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:18.148022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:18.319383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c039bc44943143c68ab1c117ce00d8ff', 'request_id': 'req-3f46971e-4911-426e-b9be-b442f2cf2689'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.327977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c039bc44943143c68ab1c117ce00d8ff', 'request_id': 'req-3f46971e-4911-426e-b9be-b442f2cf2689'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.399847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c039bc44943143c68ab1c117ce00d8ff', 'request_id': 'req-3f46971e-4911-426e-b9be-b442f2cf2689'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.416321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] The token cache is being invalidate because project c039bc44943143c68ab1c117ce00d8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:37:18.416703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.417469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.419710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 969/1968] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:37:18 2022] DELETE /identity/v3/projects/c039bc44943143c68ab1c117ce00d8ff => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:18.423552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:18.433874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b487c6c-c2c2-44a1-bcfe-f575da98998c service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:18.493461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:18.494109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:18.494394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:18.494710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] PATH_INFO: `/v3/users/374628a847c14152b0349e5781f4907e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:18.495856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=374628a847c14152b0349e5781f4907e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:18.497787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:18.511768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:18.702984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '374628a847c14152b0349e5781f4907e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.710673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '374628a847c14152b0349e5781f4907e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.726986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '374628a847c14152b0349e5781f4907e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.727753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '374628a847c14152b0349e5781f4907e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.795036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '374628a847c14152b0349e5781f4907e'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:18.802022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1000/1969] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:37:18 2022] DELETE /identity/v3/users/374628a847c14152b0349e5781f4907e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:19.260641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:19.263234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:19.285893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46971e-4911-426e-b9be-b442f2cf2689 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:19.323025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:19.359763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:19.362050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:19.362449 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:19.362910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] PATH_INFO: `/v3/projects/e11e7adfd7fe4f389171a99a90a14f08` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:19.364707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11e7adfd7fe4f389171a99a90a14f08)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:19.367194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:19.489738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11e7adfd7fe4f389171a99a90a14f08', 'request_id': 'req-6c3cc3cc-c019-4063-8850-f93bca5b00b8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:19.495268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11e7adfd7fe4f389171a99a90a14f08', 'request_id': 'req-6c3cc3cc-c019-4063-8850-f93bca5b00b8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:19.574840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11e7adfd7fe4f389171a99a90a14f08', 'request_id': 'req-6c3cc3cc-c019-4063-8850-f93bca5b00b8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:19.587862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] The token cache is being invalidate because project e11e7adfd7fe4f389171a99a90a14f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:37:19.588280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:19.589779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:19.591274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 970/1970] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:37:19 2022] DELETE /identity/v3/projects/e11e7adfd7fe4f389171a99a90a14f08 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:19.595505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:19.596971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:19.609827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431f3f7-65b8-459d-8a16-1769bcd77786 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:19.689082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:19.690116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:19.690481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:19.691126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] PATH_INFO: `/v3/users/7e2d05a3e43247d1bdd9c86f3cbf3518` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:19.693120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2d05a3e43247d1bdd9c86f3cbf3518)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:19.696755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:19.993493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2d05a3e43247d1bdd9c86f3cbf3518'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.000430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2d05a3e43247d1bdd9c86f3cbf3518'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.014837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2d05a3e43247d1bdd9c86f3cbf3518'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.015575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2d05a3e43247d1bdd9c86f3cbf3518'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.107853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-459986e9-c4ea-4b0c-95ef-3c5ccd9f80de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2d05a3e43247d1bdd9c86f3cbf3518'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.115358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1001/1971] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:37:19 2022] DELETE /identity/v3/users/7e2d05a3e43247d1bdd9c86f3cbf3518 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:20.607623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:20.609136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:20.621877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc3cc-c019-4063-8850-f93bca5b00b8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:20.682817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:20.683711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:20.683990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:20.684265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] PATH_INFO: `/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:20.685384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae2e1440fd4453e95706bc0c7f8efae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:20.687354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:20.789341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae2e1440fd4453e95706bc0c7f8efae', 'request_id': 'req-4b563109-3bd5-4663-9ba7-7dbe1c07d132'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.796467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae2e1440fd4453e95706bc0c7f8efae', 'request_id': 'req-4b563109-3bd5-4663-9ba7-7dbe1c07d132'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.925217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae2e1440fd4453e95706bc0c7f8efae', 'request_id': 'req-4b563109-3bd5-4663-9ba7-7dbe1c07d132'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.926758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:20.937919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] The token cache is being invalidate because project 2ae2e1440fd4453e95706bc0c7f8efae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:37:20.938211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.938881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b563109-3bd5-4663-9ba7-7dbe1c07d132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:37:20.940571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 971/1972] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:37:20 2022] DELETE /identity/v3/projects/2ae2e1440fd4453e95706bc0c7f8efae => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:20.946946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:20.947439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:20.947529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:20.968138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:20.969408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:20.973643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:21.031595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1002/1973] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:21.039947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.040237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.040514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.062315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:21.063581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:21.072276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 972/1974] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:21.076776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.087946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578e41e-72cc-4b1e-9986-10c8c8286bda None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.092642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.093243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.093533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.093812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.094464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:37:21.094802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.096475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.108219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1003/1975] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:37:21.112253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.121555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae160-e2ef-43fc-a50b-7e2ff00cb91e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.126486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.127130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.131067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.131067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.131067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.131067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.230470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 973/1976] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:37:21 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:37:21.234771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.244803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b251da-6df6-407d-a7c3-33254903bf3a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.305486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.306201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.306482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.306915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.307644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.309989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.429595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1004/1977] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:37:21 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:37:21.433836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.444904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fae2e-7c9e-44d1-aaf4-7efd300fd82f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.449754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.450350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.450705 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.450942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.451662 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.453174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.463780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 974/1978] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:37:21.467829 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.478106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa3eef-72a8-4f91-a182-441b61acd806 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.482882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.483494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.483774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.484049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.484699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.486306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa03131-8c83-4363-83d4-10d648bf20db admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.497739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1005/1979] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:37:21.501921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.512898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c6b05-6f01-4723-92c9-c2fe991a6fe2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.517447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.518094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.518372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.518688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] PATH_INFO: `/v3/projects/c6fc31ce382b4d5bbe38a82baceef2c0/users/198bee88d5214576af4187d833f90e2c/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.533524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fc31ce382b4d5bbe38a82baceef2c0, user_id=198bee88d5214576af4187d833f90e2c, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.536435 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.611853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 975/1980] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:37:21 2022] PUT /identity/v3/projects/c6fc31ce382b4d5bbe38a82baceef2c0/users/198bee88d5214576af4187d833f90e2c/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:37:21.623420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ea21-ca39-4b09-a029-c340214c408c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.623862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ea21-ca39-4b09-a029-c340214c408c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.624187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ea21-ca39-4b09-a029-c340214c408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.656095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-aca1ea21-ca39-4b09-a029-c340214c408c None None] MFA Rules not processed for user `198bee88d5214576af4187d833f90e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:21.657320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-aca1ea21-ca39-4b09-a029-c340214c408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:21.725623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1006/1981] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:37:21.738731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:37:21.740447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:21.799741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1692969-bdd2-43bb-96b3-dbb656e5c7f6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:37:21.803418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:37:21.804169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:21.804559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:21.804723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:21.813641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:37:21.815102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:37:21.824666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 976/1982] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:37:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:37:24.492502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7860cb-7dc3-47f8-b4de-58c6aca64bae None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:37:24.492964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7860cb-7dc3-47f8-b4de-58c6aca64bae None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:37:24.493149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7860cb-7dc3-47f8-b4de-58c6aca64bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:37:24.502576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:24.516018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4e7860cb-7dc3-47f8-b4de-58c6aca64bae None None] MFA Rules not processed for user `020ff888a8a44d2eb6961829407637a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:37:24.517250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4e7860cb-7dc3-47f8-b4de-58c6aca64bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:37:24.574105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1007/1983] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:37:31.820322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:37:34.668548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=23 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:08.915614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:08.917094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:09.001881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adbe05-c60a-4f8b-b2a5-9d58d96204c1 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:09.007082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:09.007721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:09.008002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:09.008274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:09.010038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:09.021755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:09.023432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:09.033220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 977/1984] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:10.843552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeff3e-efe3-4e5c-86f9-729612cd89af None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:10.844023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeff3e-efe3-4e5c-86f9-729612cd89af None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:10.844133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeff3e-efe3-4e5c-86f9-729612cd89af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:10.863508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-aaeeff3e-efe3-4e5c-86f9-729612cd89af None None] MFA Rules not processed for user `b69eb168c92343b5819d4795eb85dd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:10.864711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-aaeeff3e-efe3-4e5c-86f9-729612cd89af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:10.920731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1008/1985] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:15.965597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:15.967162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:16.055769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2c7c8-7765-4c51-8e09-22b027719259 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:16.060614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:16.061336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:16.061564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:16.061840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] PATH_INFO: `/v3/users/9a40ba66617b4256aa59e45785976096` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:16.062990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a40ba66617b4256aa59e45785976096)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:16.064893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:16.207290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a40ba66617b4256aa59e45785976096'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:16.208053 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a40ba66617b4256aa59e45785976096'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:16.215310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a40ba66617b4256aa59e45785976096'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:16.228361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a40ba66617b4256aa59e45785976096'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:16.289913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a40ba66617b4256aa59e45785976096'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:16.296480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 978/1986] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:38:15 2022] DELETE /identity/v3/users/9a40ba66617b4256aa59e45785976096 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:38:16.300981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:16.301743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:16.301743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:16.321125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:16.322322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:16.386958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1009/1987] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:38:16.397087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:16.398845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:16.410713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:16.459535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ba7bf-2bb1-4843-be26-85563b722ade admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:16.463399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:16.464102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:16.464486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:16.464825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:16.474311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:16.476310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:16.486487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 979/1988] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:17.288913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:17.290350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:17.302769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c755b09-4463-499b-96c9-d505ac952f1c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:17.309059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:17.309972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:17.310441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:17.311002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] PATH_INFO: `/v3/projects/ec51c4d2533742909b2dc6e87d01ec74` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:17.312330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec51c4d2533742909b2dc6e87d01ec74)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:17.314431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:17.429969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec51c4d2533742909b2dc6e87d01ec74', 'request_id': 'req-5549b287-099b-4b78-85bc-3be17ecdbea7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.436666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec51c4d2533742909b2dc6e87d01ec74', 'request_id': 'req-5549b287-099b-4b78-85bc-3be17ecdbea7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.512556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec51c4d2533742909b2dc6e87d01ec74', 'request_id': 'req-5549b287-099b-4b78-85bc-3be17ecdbea7'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.531445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] The token cache is being invalidate because project ec51c4d2533742909b2dc6e87d01ec74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:38:17.532269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.533271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.536014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1010/1989] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:38:17 2022] DELETE /identity/v3/projects/ec51c4d2533742909b2dc6e87d01ec74 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:17.540424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:17.557652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebede3b-1fd5-441d-84d7-b51a9039a3e0 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:17.622555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:17.623250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:17.623533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:17.623808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] PATH_INFO: `/v3/users/791d1a0bfda24fc68adeee8a6d11ca5b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:17.625067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791d1a0bfda24fc68adeee8a6d11ca5b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:17.627026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:17.811016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791d1a0bfda24fc68adeee8a6d11ca5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.811747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791d1a0bfda24fc68adeee8a6d11ca5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.822093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791d1a0bfda24fc68adeee8a6d11ca5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.836756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791d1a0bfda24fc68adeee8a6d11ca5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.912403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-763d541e-8984-4da1-adbb-73b151e48612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791d1a0bfda24fc68adeee8a6d11ca5b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:17.923657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 980/1990] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:38:17 2022] DELETE /identity/v3/users/791d1a0bfda24fc68adeee8a6d11ca5b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:18.401203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:18.402926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:18.421952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549b287-099b-4b78-85bc-3be17ecdbea7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:18.453766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:18.487018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:18.487018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:18.487267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:18.487603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] PATH_INFO: `/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:18.488659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2aa99b8a3a4a80a2a578a6f3bc29cb)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:18.490635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:18.583456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2aa99b8a3a4a80a2a578a6f3bc29cb', 'request_id': 'req-6e14adcf-694c-42e8-8429-a4c4b72b3e83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:18.589704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2aa99b8a3a4a80a2a578a6f3bc29cb', 'request_id': 'req-6e14adcf-694c-42e8-8429-a4c4b72b3e83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:18.632870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2aa99b8a3a4a80a2a578a6f3bc29cb', 'request_id': 'req-6e14adcf-694c-42e8-8429-a4c4b72b3e83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:18.646839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] The token cache is being invalidate because project 0c2aa99b8a3a4a80a2a578a6f3bc29cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:38:18.652774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:18.652774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6e14adcf-694c-42e8-8429-a4c4b72b3e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:18.652774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1011/1991] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:38:18 2022] DELETE /identity/v3/projects/0c2aa99b8a3a4a80a2a578a6f3bc29cb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:18.655142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:18.656382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:18.656680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:18.678873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:18.680012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:18.708323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:18.742543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 981/1992] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:38:18.749759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:18.750049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:18.750369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:18.770599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:18.771814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:18.780621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1012/1993] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:18.784800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:18.794641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6996b07-9ed5-4d77-bd74-aab3375f3d50 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:18.799447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:18.800062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:18.800344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:18.800621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:18.801275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:38:18.801590 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:18.803260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:18.817664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 982/1994] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:38:18.822668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:18.832049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a57af-7a73-47c9-bb6c-d08cd0b265a7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:18.837403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:18.838026 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:18.840973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:18.840973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:18.840973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:18.841347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:18.938490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1013/1995] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:18 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:38:18.942419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:18.953245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4f9da-5b43-46ef-a1b3-315889aada01 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.010741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.011364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.011650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.011925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.012566 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.014573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.081608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 983/1996] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:18 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:19.082937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.094179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed6fce-8a8f-4156-a2e5-3ae95ee85535 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.098956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.099588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.100104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.100222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.100780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.102380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.115077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1014/1997] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:19.117853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.127858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b308a-9fb2-4b9c-ae70-dec61af71837 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.133032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.133657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.133934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.134205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.134950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.136527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d67ce8-4ec9-4364-9c85-e6d3779c2103 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.147008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 984/1998] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:19.151170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.161177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9be9-7749-467d-8558-1c874d46b529 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.165757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.166402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.166799 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.167077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] PATH_INFO: `/v3/projects/e23baba5afc04e05b09fef005bb1d6d5/users/e73c0f3bc4e8442eabfe603efdca73fb/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.182230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23baba5afc04e05b09fef005bb1d6d5, user_id=e73c0f3bc4e8442eabfe603efdca73fb, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.185014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.265804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1015/1999] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:19 2022] PUT /identity/v3/projects/e23baba5afc04e05b09fef005bb1d6d5/users/e73c0f3bc4e8442eabfe603efdca73fb/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:38:19.276117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.276407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.276768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.305265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] MFA Rules not processed for user `e73c0f3bc4e8442eabfe603efdca73fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:19.306472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:19.369017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 985/2000] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:19.373381 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.431962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:19.437039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2287c-dde6-4d58-8c45-ef19aa6610b9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.441870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.447076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.447076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.447076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.447076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.447076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.525379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1016/2001] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:19 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:38:19.534304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.545558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d4c86-b036-4b4f-b4e9-b899b4edb416 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.571109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:19.605395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.606007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.609112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.609112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.609112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.609112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.677673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 986/2002] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:19 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:19.682773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.693102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb5c7b-4c0b-4a68-a4b9-f6891164b653 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.697749 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.698352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.698675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.698951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.699622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.701215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.712006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1017/2003] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:19.716530 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.727910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea65de-6444-4862-a87e-78952d46d7d0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.732499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.733133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.733415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.733689 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] PATH_INFO: `/v3/projects/584c2ccafd8744e9bccf04ca8b48877e/users/48c6fd45fb9340c993ac6c2eab28b7ae/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.748231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=584c2ccafd8744e9bccf04ca8b48877e, user_id=48c6fd45fb9340c993ac6c2eab28b7ae, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.750951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.826109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 987/2004] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:19 2022] PUT /identity/v3/projects/584c2ccafd8744e9bccf04ca8b48877e/users/48c6fd45fb9340c993ac6c2eab28b7ae/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:19.830001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.840666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2c7a-b6fc-42a3-be9c-a4f48415d026 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.896272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.896890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.897173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.897945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.898624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.900215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab6fdf-8aeb-4e3a-9f76-787202ca63a1 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.909839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1018/2005] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:19.914939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:19.925391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104c8f4-b067-440f-b991-264c058328cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:19.930028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:19.930712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.930992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.931266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] PATH_INFO: `/v3/projects/584c2ccafd8744e9bccf04ca8b48877e/users/48c6fd45fb9340c993ac6c2eab28b7ae/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.933442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584c2ccafd8744e9bccf04ca8b48877e, user_id=48c6fd45fb9340c993ac6c2eab28b7ae, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:19.936130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:19.946342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 988/2006] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:19 2022] PUT /identity/v3/projects/584c2ccafd8744e9bccf04ca8b48877e/users/48c6fd45fb9340c993ac6c2eab28b7ae/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:19.957122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:19.957418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:19.957701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:19.987157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] MFA Rules not processed for user `48c6fd45fb9340c993ac6c2eab28b7ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:19.988365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:20.061039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1019/2007] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:20.072779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:20.076401 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:20.132846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fb251-fc38-457e-9a3c-e5e6471d65a0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:20.136470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:20.137076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:20.137351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:20.137619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:20.146801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:20.148836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:20.158419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 989/2008] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:20.338111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:20.348148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a616c8-1e00-4951-8053-37325a534972 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:20.351778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:20.352382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:20.352663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:20.352937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:20.399356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:20.408815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:20.410289 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920b932-9b72-4210-b4a7-88104d84d2b5 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:20.419354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1020/2009] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:30.154020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:30.483334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:38.684338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:38.685744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:38.750040 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900a40d-dde0-4791-86d6-870b2a5ee566 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:38.754497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:38.755146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:38.755422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:38.755692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] PATH_INFO: `/v3/users/2e2cb54b72f94223b060711c1fb6828b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:38.756790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2cb54b72f94223b060711c1fb6828b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:38.758717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:38.962706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2cb54b72f94223b060711c1fb6828b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:38.962706 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2cb54b72f94223b060711c1fb6828b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:38.969485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2cb54b72f94223b060711c1fb6828b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:38.982489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2cb54b72f94223b060711c1fb6828b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.037325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2cb54b72f94223b060711c1fb6828b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.043860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 990/2010] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:38:38 2022] DELETE /identity/v3/users/2e2cb54b72f94223b060711c1fb6828b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:39.048366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:39.048856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:39.048952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:39.067788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:39.071876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:39.127571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1021/2011] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:39.137984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:39.139454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:39.202402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb051a-2991-47b3-bafe-5a1bb3baa109 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:39.210556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:39.211172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:39.211533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:39.211767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:39.221062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:39.222601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596f02-5c80-40c2-aaab-7e803ca50c47 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:39.232047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 991/2012] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:38:39.628320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:39.630259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:39.657061 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841fb0f-aadf-43ea-b7a8-ed997b1dfbea None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:39.673365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:39.674262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:39.674651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:39.674995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] PATH_INFO: `/v3/projects/3df9b05f75e94bcb9d9a16975f70ca97` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:39.676192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df9b05f75e94bcb9d9a16975f70ca97)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:39.678872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:39.779612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df9b05f75e94bcb9d9a16975f70ca97', 'request_id': 'req-79444976-9b0f-43b6-8864-9658f6fa3836'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.785147 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df9b05f75e94bcb9d9a16975f70ca97', 'request_id': 'req-79444976-9b0f-43b6-8864-9658f6fa3836'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.860243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df9b05f75e94bcb9d9a16975f70ca97', 'request_id': 'req-79444976-9b0f-43b6-8864-9658f6fa3836'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.872117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] The token cache is being invalidate because project 3df9b05f75e94bcb9d9a16975f70ca97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:38:39.878036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.878036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-79444976-9b0f-43b6-8864-9658f6fa3836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:39.878036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1022/2013] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:38:39 2022] DELETE /identity/v3/projects/3df9b05f75e94bcb9d9a16975f70ca97 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:39.879388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:39.879921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:39.879921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:39.900297 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:39.901501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:39.918548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:39.962421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 992/2014] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:39.970415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:39.970745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:39.971049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:39.991213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:39.992432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:40.003897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1023/2015] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:40.005546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.015945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbf2ae-a1db-4cb5-92f9-9fcd268fa334 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.020703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.021317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.021627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.021928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.022637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:38:40.022944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.024539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.038561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 993/2016] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:38:40.041235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.050637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f7364-938c-44da-8e92-f49121de64f4 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.055916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.056808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.057085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.057357 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.058004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.059929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.152230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1024/2017] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:40 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:38:40.156446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.166331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bb5cd-4489-41a3-90de-05f1bc1d0514 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.223823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.224515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.224807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.225077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.225696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.227632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.306828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 994/2018] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:40 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:38:40.310924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.321305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf3a33-cbbf-45fd-8efb-f86e8b0c672f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.325948 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.326580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.326865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.327140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.327786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.329375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.339628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1025/2019] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:40.343374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.354292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9735a-5972-46f7-b9bb-359b30cb80c4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.359146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.359749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.360082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.360359 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.361002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.362608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71531-be38-4758-b593-f4ebe38da7aa admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.374398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 995/2020] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:38:40.378392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.388393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdd3b-99c1-45cb-aeab-7a64b4281d57 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.392976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.393618 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.393901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.394178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] PATH_INFO: `/v3/projects/2c7e9f025f5044f48e697f446cb09c95/users/18cd48c7eedd4bd6a32330fe914a48db/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.408329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7e9f025f5044f48e697f446cb09c95, user_id=18cd48c7eedd4bd6a32330fe914a48db, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.411008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.511092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1026/2021] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:40 2022] PUT /identity/v3/projects/2c7e9f025f5044f48e697f446cb09c95/users/18cd48c7eedd4bd6a32330fe914a48db/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:38:40.522794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.524994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.524994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.551379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] MFA Rules not processed for user `18cd48c7eedd4bd6a32330fe914a48db`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:40.552589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:40.618723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 996/2022] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:40.629911 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:40.631646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:40.654664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:40.690719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43d33-af5c-430c-89bb-1a5634baee2e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:40.694550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:40.695171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:40.695456 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:40.695733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:40.705360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:40.706827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ff716-d89d-4395-9c99-9d859a2c1f71 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:40.716332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1027/2023] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:47.188684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:47.190230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:47.211344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:47.265609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96eb6a-a15a-47a0-920a-38f132e2125a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:47.270058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:47.270750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:47.271029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:47.271305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] PATH_INFO: `/v3/users/e73c0f3bc4e8442eabfe603efdca73fb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:47.273102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73c0f3bc4e8442eabfe603efdca73fb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:47.275173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:47.441664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73c0f3bc4e8442eabfe603efdca73fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:47.442599 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73c0f3bc4e8442eabfe603efdca73fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:47.450541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73c0f3bc4e8442eabfe603efdca73fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:47.465988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73c0f3bc4e8442eabfe603efdca73fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:47.539780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73c0f3bc4e8442eabfe603efdca73fb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:47.545948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 997/2024] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:38:47 2022] DELETE /identity/v3/users/e73c0f3bc4e8442eabfe603efdca73fb => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:47.550615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:47.551064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:47.551237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:47.571369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:47.572567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:47.629428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1028/2025] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:47.645361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:47.646894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:47.705033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb90ac3-7825-417f-aa2b-d086b7ec0ab1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:47.708534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:47.709135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:47.709412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:47.709682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:47.718620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:47.720039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:47.729561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 998/2026] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:38:48.153373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:48.155034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:48.168887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0045ea-9937-41f0-b427-5481f92aaa73 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:48.173470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:48.174102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:48.174379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:48.174687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] PATH_INFO: `/v3/projects/e23baba5afc04e05b09fef005bb1d6d5` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:48.175808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23baba5afc04e05b09fef005bb1d6d5)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:48.177790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:48.289473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23baba5afc04e05b09fef005bb1d6d5', 'request_id': 'req-cc287952-9ce5-4ed1-8386-90aa8b87312c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.294912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23baba5afc04e05b09fef005bb1d6d5', 'request_id': 'req-cc287952-9ce5-4ed1-8386-90aa8b87312c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.345335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23baba5afc04e05b09fef005bb1d6d5', 'request_id': 'req-cc287952-9ce5-4ed1-8386-90aa8b87312c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.357301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] The token cache is being invalidate because project e23baba5afc04e05b09fef005bb1d6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:38:48.357591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.358232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.359766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1029/2027] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:38:48 2022] DELETE /identity/v3/projects/e23baba5afc04e05b09fef005bb1d6d5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:48.365808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:48.378851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0eed5-ff06-4ea2-a924-fc18b264f3ba service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:48.416820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:48.436383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:48.437023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:48.437308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:48.437586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] PATH_INFO: `/v3/users/48c6fd45fb9340c993ac6c2eab28b7ae` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:48.438856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c6fd45fb9340c993ac6c2eab28b7ae)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:48.440809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:48.657593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c6fd45fb9340c993ac6c2eab28b7ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.658252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c6fd45fb9340c993ac6c2eab28b7ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.665203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c6fd45fb9340c993ac6c2eab28b7ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.677228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c6fd45fb9340c993ac6c2eab28b7ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.745603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-90ab134c-3d95-4ce5-8a18-ad710d417c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c6fd45fb9340c993ac6c2eab28b7ae'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:48.751952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 999/2028] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:38:48 2022] DELETE /identity/v3/users/48c6fd45fb9340c993ac6c2eab28b7ae => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:49.215753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.217222 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:49.230269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc287952-9ce5-4ed1-8386-90aa8b87312c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.279288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:49.292652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.293293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.293573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.293846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] PATH_INFO: `/v3/projects/584c2ccafd8744e9bccf04ca8b48877e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.294992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584c2ccafd8744e9bccf04ca8b48877e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.296893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.420508 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584c2ccafd8744e9bccf04ca8b48877e', 'request_id': 'req-5316bd76-a6a3-481d-9876-5f31eb14c8c5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:49.426098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584c2ccafd8744e9bccf04ca8b48877e', 'request_id': 'req-5316bd76-a6a3-481d-9876-5f31eb14c8c5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:49.477109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584c2ccafd8744e9bccf04ca8b48877e', 'request_id': 'req-5316bd76-a6a3-481d-9876-5f31eb14c8c5'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:49.488638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] The token cache is being invalidate because project 584c2ccafd8744e9bccf04ca8b48877e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:38:49.488924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:49.489574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5316bd76-a6a3-481d-9876-5f31eb14c8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:38:49.491739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1030/2029] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:38:49 2022] DELETE /identity/v3/projects/584c2ccafd8744e9bccf04ca8b48877e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:49.498748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.499213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.502265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.519863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:49.521066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:49.578859 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1000/2030] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:38:49.586732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.587057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.587339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.607299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:49.608509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:49.616555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1031/2031] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:49.621069 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.630471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfdeaf-81d0-47bf-b531-2b88e2f5cf1c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.635052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.635647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.635920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.636195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.636832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:38:49.637130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.638721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.650460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1001/2032] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:38:49.654718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.663953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f953416-7ce4-4d69-957b-93151f4b4816 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.669105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.669874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.669979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.670205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.670933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.672740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.750702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1032/2033] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:49 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:49.751902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.762831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11626013-3227-4167-8196-cd8593cfbc09 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.807666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:49.820336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.820937 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.821215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.821484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.822146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.824109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.888503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1002/2034] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:49 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:49.892670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.903457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0f3da-8ee1-40ea-a31f-851d6d5b07cc admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.908287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.908979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.909274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.909530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.910166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.911801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.922309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1033/2035] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:49.926174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.936775 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86c106-9e68-4ad5-9be6-a3025e612610 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.941552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.942147 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.942427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.942738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.943404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.945007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2af79-d6ae-4b2c-bdc5-5fd75475013f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:49.955490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1003/2036] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:49.959580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:49.969873 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b425de-b170-4db5-adce-94acba454275 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:49.974708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:49.975350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:49.975638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:49.975913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] PATH_INFO: `/v3/projects/19fe796def664183a9867d58cf463d19/users/afd61028a8804fba91eee0cab208e512/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:49.990937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fe796def664183a9867d58cf463d19, user_id=afd61028a8804fba91eee0cab208e512, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:49.993567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.054960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1034/2037] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:49 2022] PUT /identity/v3/projects/19fe796def664183a9867d58cf463d19/users/afd61028a8804fba91eee0cab208e512/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:38:50.079653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.079965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.080244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.108856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] MFA Rules not processed for user `afd61028a8804fba91eee0cab208e512`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:50.110074 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:50.176369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1004/2038] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:50.181164 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.241439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053d7fc-4b08-4c82-9509-ff3f295d9a0c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.246069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.246701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.246986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.247259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.247875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.249757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.305482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:50.312502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1035/2039] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:50.316572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.327647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a278e83-5deb-483a-9ba8-ce394e3c7b07 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.383497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.384136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.384415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.384688 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.385318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.387242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.459710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1005/2040] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:38:50.463643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.473770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4ba0-9241-467f-bddd-797c8f6f24c7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.478463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.479086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.479384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.479644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.480284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.481912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.492283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1036/2041] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:50.496656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.506666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b8ac-a02b-4881-a173-b6d248ac023f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.512038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.512643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.512925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.513202 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.513840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.515546 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683314c-02d4-4446-b27f-2422af6746ab admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.526629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1006/2042] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:50.530958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.540921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76527cac-9542-4014-9288-7f5922a5f354 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.545578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.546221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.546528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.546809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] PATH_INFO: `/v3/projects/33d9cf8e6dfe4c40a20ee88847d5da00/users/19a3ff0c4e4e4e329a54d00479311eaa/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.561883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d9cf8e6dfe4c40a20ee88847d5da00, user_id=19a3ff0c4e4e4e329a54d00479311eaa, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.564559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.631431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1037/2043] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:50 2022] PUT /identity/v3/projects/33d9cf8e6dfe4c40a20ee88847d5da00/users/19a3ff0c4e4e4e329a54d00479311eaa/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:38:50.642423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.642730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.643030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.671667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] MFA Rules not processed for user `19a3ff0c4e4e4e329a54d00479311eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:50.675492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:50.693963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:50.737777 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1007/2044] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:50.742194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.803698 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32bd49-d23a-4911-bd1c-e666833b8ca4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.809107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.809730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.810010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.810281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.810939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.812826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:50.876130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1038/2045] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:50.877433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:50.888961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d4b28-b0eb-4343-852c-7a38c7476643 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:50.947095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:50.947746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:50.948026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:50.948298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:50.948931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:50.950878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.025939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1008/2046] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:38:50 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:38:51.029938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:51.040917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416332dc-2473-499c-b3b7-0e39bbd88310 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:51.045547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:51.046142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.046421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.046726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.047361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:51.048950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.060557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1039/2047] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:51.064949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:51.075858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6826a-fecb-472f-80a0-4a9bf4918e27 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:51.080094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:51.080745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.081024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.081293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] PATH_INFO: `/v3/projects/dad70140903445048f45d71faf746ffb/users/178e55e7c1bc4091a0635d36524c8a5c/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.097189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad70140903445048f45d71faf746ffb, user_id=178e55e7c1bc4091a0635d36524c8a5c, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:51.099848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.197661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1009/2048] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:51 2022] PUT /identity/v3/projects/dad70140903445048f45d71faf746ffb/users/178e55e7c1bc4091a0635d36524c8a5c/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:51.210169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:51.220839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038195dd-c2ea-426d-b434-acde3d9c1445 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:51.276315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:51.276940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.277110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.277497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.278130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:51.279743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5eb51-ec82-4156-af9c-1724f9201f60 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.289214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1040/2049] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:38:51.294328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:51.304779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e624-018b-4836-abd8-5c6b5c7e75ef admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:51.309760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:51.310427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.310735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.311010 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] PATH_INFO: `/v3/projects/dad70140903445048f45d71faf746ffb/users/178e55e7c1bc4091a0635d36524c8a5c/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.312883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad70140903445048f45d71faf746ffb, user_id=178e55e7c1bc4091a0635d36524c8a5c, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:51.315522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.325443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1010/2050] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:38:51 2022] PUT /identity/v3/projects/dad70140903445048f45d71faf746ffb/users/178e55e7c1bc4091a0635d36524c8a5c/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:38:51.335320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.335612 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.335896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.365084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] MFA Rules not processed for user `178e55e7c1bc4091a0635d36524c8a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:38:51.366274 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:51.385693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:51.432922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1041/2051] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:38:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:38:51.444188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:38:51.445919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:38:51.503479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1c9b3-764d-493e-815f-5144b2350a86 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:38:51.507115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:38:51.507703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:38:51.507974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:38:51.508244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:38:51.537430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:38:51.563909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:38:51.565329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:38:51.574990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1011/2052] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:38:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:00.878223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:00.888511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b08470-ad4d-4925-84ae-0236e41a3576 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:00.892048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:00.892650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:00.892930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:00.893201 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:00.954088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:00.955633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1cf9e-a89d-4c30-8323-8de6903fda4d service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:00.964632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1042/2053] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:01.608237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=14 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:05.408569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:05.410186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:05.421780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6263e-5c4e-4927-b25a-aab0aba356ef service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:05.425370 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:05.425968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:05.426244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:05.426539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:05.436268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:05.437694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:05.447388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1012/2054] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:10.960473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=47 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:11.957531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:11.957835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:11.958111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:11.981843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:11.983252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:12.055621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1043/2055] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:12.097539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:12.099037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:12.168950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2f537-d0f6-4af2-b6ab-b66e5167aa59 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:12.174365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:12.175006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:12.175285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:12.175557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:12.184694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:12.186445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:12.195575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1013/2056] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:39:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:39:14.015204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:14.016662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.083621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f5e45-c805-4a09-8a87-11dcd51a2d74 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:14.088071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:14.093019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:14.093019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:14.093019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] PATH_INFO: `/v3/users/198bee88d5214576af4187d833f90e2c` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:14.093019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] RBAC: Authorizing `identity:delete_user(user_id=198bee88d5214576af4187d833f90e2c)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:14.093019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:14.279076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198bee88d5214576af4187d833f90e2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.290037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198bee88d5214576af4187d833f90e2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.325254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198bee88d5214576af4187d833f90e2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.326116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198bee88d5214576af4187d833f90e2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.330361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:14.337626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.348140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583624-6055-47ec-b186-0a1dc5bb21b2 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:14.352996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:14.353754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:14.354142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:14.354547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] PATH_INFO: `/v3/users/18cd48c7eedd4bd6a32330fe914a48db` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:14.355858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cd48c7eedd4bd6a32330fe914a48db)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:14.357880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:14.413846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5d7fb4e1-0a47-41ca-8896-a2f51f99ecec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198bee88d5214576af4187d833f90e2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.421974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1044/2057] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:14 2022] DELETE /identity/v3/users/198bee88d5214576af4187d833f90e2c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:14.426227 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:14.426488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:14.426907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:14.448330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:14.449591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.471474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:14.519951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1045/2058] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:14.531058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:14.533712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.585025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cd48c7eedd4bd6a32330fe914a48db'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.586111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cd48c7eedd4bd6a32330fe914a48db'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.595803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cd48c7eedd4bd6a32330fe914a48db'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.614081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cd48c7eedd4bd6a32330fe914a48db'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.621825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07503439-dd0d-439f-9f82-7538d62e96c8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:14.627637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:14.628323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:14.628608 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:14.629316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:14.633012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.654912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:14.657815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4111-dadf-44ae-978e-a2f718be2462 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:14.679029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cd48c7eedd4bd6a32330fe914a48db'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:14.706785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1046/2059] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:14.720392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1014/2060] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:14 2022] DELETE /identity/v3/users/18cd48c7eedd4bd6a32330fe914a48db => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:14.722454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:14.722982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:14.723539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:14.775497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:14.778808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:14.934548 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1047/2061] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:14.944312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:14.946054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:15.071152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:15.076395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e586-bf12-4f83-a9ea-90eaae3e7e23 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:15.083753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:15.084400 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.084696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.085015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.099048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:15.101146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890ba39-6fc9-4676-8d5e-024fd481ff31 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:15.122550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1015/2062] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:15.213254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:15.213254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:15.225466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120b4cc-a763-405d-ad22-77fdd4ed4533 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:15.237304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:15.239438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.241718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.242423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] PATH_INFO: `/v3/projects/c6fc31ce382b4d5bbe38a82baceef2c0` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.245885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fc31ce382b4d5bbe38a82baceef2c0)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:15.251927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:15.398786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fc31ce382b4d5bbe38a82baceef2c0', 'request_id': 'req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.413724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fc31ce382b4d5bbe38a82baceef2c0', 'request_id': 'req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.417794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:15.494657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fc31ce382b4d5bbe38a82baceef2c0', 'request_id': 'req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.513073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] The token cache is being invalidate because project c6fc31ce382b4d5bbe38a82baceef2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:15.513305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.514223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec05f9f2-a1e8-472d-8b4c-dda9036c60c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.516693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1048/2063] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:15 2022] DELETE /identity/v3/projects/c6fc31ce382b4d5bbe38a82baceef2c0 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:15.521620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.522272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.522272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.552021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:15.553781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:15.638978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1016/2064] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:39:15.641807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.642124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.642405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.674953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:15.677334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:15.691178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1049/2065] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:15.700679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:15.706545 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042011c-77d6-4784-9327-565a1f5eca27 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:15.711352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:15.711968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.712248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.712522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.713172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:39:15.713470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:15.727209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:15.727571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:15.740983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1017/2066] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:39:15.741310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:15.750839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fd8f8-30a2-4004-b34e-24ae9ee2a044 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:15.752355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:15.770336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:15.770336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.770336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.770336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] PATH_INFO: `/v3/projects/2c7e9f025f5044f48e697f446cb09c95` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.771074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7e9f025f5044f48e697f446cb09c95)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:15.771295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12950-7e8b-461d-93af-c07dbbb1cb17 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:15.771383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:15.779696 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:15.780772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:15.780968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:15.781869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:15.782582 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:15.785168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:15.907367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1018/2067] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:15 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:15.914713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:15.919268 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7e9f025f5044f48e697f446cb09c95', 'request_id': 'req-81630797-cc78-4499-a311-5271c24ab27f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:15.946439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7e9f025f5044f48e697f446cb09c95', 'request_id': 'req-81630797-cc78-4499-a311-5271c24ab27f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.032275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eff66-dcea-45b2-bff9-c342e9d09aa9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.037188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.037807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.038087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.038363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.039042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.040965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.059531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7e9f025f5044f48e697f446cb09c95', 'request_id': 'req-81630797-cc78-4499-a311-5271c24ab27f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.079578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] The token cache is being invalidate because project 2c7e9f025f5044f48e697f446cb09c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:16.080029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.080944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-81630797-cc78-4499-a311-5271c24ab27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.083075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1050/2068] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:15 2022] DELETE /identity/v3/projects/2c7e9f025f5044f48e697f446cb09c95 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:16.087828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf64c4-f635-4084-ae5b-490019915276 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.088854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf64c4-f635-4084-ae5b-490019915276 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.088854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf64c4-f635-4084-ae5b-490019915276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.112658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1019/2069] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:15 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:16.116589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.118540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.127874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1bbf64c4-f635-4084-ae5b-490019915276 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:16.129093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1bbf64c4-f635-4084-ae5b-490019915276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.134944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2760b21-ba7b-4b03-8e26-72e24870b331 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.138195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1051/2070] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:16.152875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.152875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.152875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.152875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.153358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.154622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.155028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.155379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.161581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.182109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1020/2071] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:16.186300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.197439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084e942-0ad6-4182-84dd-de5bc8435046 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.203115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.203115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.203848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.204171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.204813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.206727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.217634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:16.218601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1021/2072] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:39:16.220921 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.222604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.235808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e15fd-57c2-460f-9bd2-214d077cc59d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.236707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1052/2073] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:16.238045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.239896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.240570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.240994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.241274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] PATH_INFO: `/v3/projects/56e0bbf31a4148cc8fa2264d0d0de4ac/users/5cb0b0bbc0d448c89c7d9cd606c63191/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.251447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8098b2-8e07-4957-a19d-e316b595d387 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.260515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e0bbf31a4148cc8fa2264d0d0de4ac, user_id=5cb0b0bbc0d448c89c7d9cd606c63191, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.260679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.260679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.260679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.260679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.261006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:39:16.261088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc18c1-d52d-408b-8633-7a65048672f8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.261639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.263889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.290674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1053/2074] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:39:16.293515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.307437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2daf17-88c5-4570-a33b-395e481908df admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.320424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.321616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.322089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.322420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.323109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.327932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:16.328140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.339960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1022/2075] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:16 2022] PUT /identity/v3/projects/56e0bbf31a4148cc8fa2264d0d0de4ac/users/5cb0b0bbc0d448c89c7d9cd606c63191/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:16.356588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.358026 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.358562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.424447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] MFA Rules not processed for user `5cb0b0bbc0d448c89c7d9cd606c63191`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:16.429493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1054/2076] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:39:16.430683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.440552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.519445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5e4d8-3157-4bec-bd07-c2c75cc0b20d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.525309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.526112 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.526441 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.527108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.527987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.529965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.549219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1023/2077] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:16.561424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.564096 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.617614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1055/2078] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:16.618862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.629531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da9b05-00dd-4a29-8a44-8d52d38f8fe8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.634863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.635798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.636092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.636405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.637144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.641219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.644881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:16.654440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3a4c-13fe-4adc-9dfc-774e1ea5773f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.658324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1056/2079] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:39:16.659370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.663322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.665957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.666282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.666583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.673262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c9aa4-b80d-4be4-b3c7-4fb90384837e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.681904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.681904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.682369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.682369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.682369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.682369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.682711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.684209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.693183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1024/2080] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:39:16.697731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1057/2081] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:16.705131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.730960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.735715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012d474-3a5e-4715-a6c7-d59d72307ccb service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.745392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.745392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.746761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.747251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] PATH_INFO: `/v3/projects/7a1c12d9a4454de59ed63d3cedb71a58/users/e7a6ae7333d2422ebc9f616d482e527a/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.754697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b237f94-82e3-4a66-bca5-60a2e33c6b38 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:16.760172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:16.761213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.766818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.766818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] PATH_INFO: `/v3/users/5cb0b0bbc0d448c89c7d9cd606c63191` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.766818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb0b0bbc0d448c89c7d9cd606c63191)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.766818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.784333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1c12d9a4454de59ed63d3cedb71a58, user_id=e7a6ae7333d2422ebc9f616d482e527a, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:16.788284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198c2a7-e54a-4dfa-b956-daada67fc6a9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:16.864639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1025/2082] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:16 2022] PUT /identity/v3/projects/7a1c12d9a4454de59ed63d3cedb71a58/users/e7a6ae7333d2422ebc9f616d482e527a/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:16.866941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:16.867328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:16.867664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:16.889516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:16.891462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.961454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1026/2083] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:39:16.967346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb0b0bbc0d448c89c7d9cd606c63191'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.974161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:16.975607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:16.976491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb0b0bbc0d448c89c7d9cd606c63191'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.989781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:16.992603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb0b0bbc0d448c89c7d9cd606c63191'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:16.993484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb0b0bbc0d448c89c7d9cd606c63191'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:17.053584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836cde3-c006-45c1-a807-758e5c3d2be1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:17.059899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-2ae1dc56-3f90-49f9-86aa-3c83f6d6fec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb0b0bbc0d448c89c7d9cd606c63191'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:17.061631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:17.067847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1058/2084] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:16 2022] DELETE /identity/v3/users/5cb0b0bbc0d448c89c7d9cd606c63191 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:17.071466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:17.071466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:17.071466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:17.073363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:17.073363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:17.073684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:17.079063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:17.102477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:17.104717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:17.105875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:17.108143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:17.117557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1027/2085] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:17.209099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1059/2086] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:39:17.210254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:17.212219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:17.232761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e92501-cc20-4c07-8592-0183195a8363 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:17.330290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:17.330918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:17.331190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:17.331468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:17.344001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:17.346209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:17.358848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1028/2087] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:39:18.034430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:18.038040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:18.062463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24198334-ff4f-4252-b2d8-25ecfc700513 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:18.072348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:18.073655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:18.076495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:18.078797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] PATH_INFO: `/v3/users/afd61028a8804fba91eee0cab208e512` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:18.080383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd61028a8804fba91eee0cab208e512)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:18.082582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:18.241524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:18.241524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:18.266745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4fa8b-0fb9-4518-b93f-4dca998c68b7 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:18.273084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:18.273887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:18.274373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:18.274808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] PATH_INFO: `/v3/projects/56e0bbf31a4148cc8fa2264d0d0de4ac` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:18.276044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e0bbf31a4148cc8fa2264d0d0de4ac)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:18.278727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:18.390826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd61028a8804fba91eee0cab208e512'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.397630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:18.401223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd61028a8804fba91eee0cab208e512'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.418818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd61028a8804fba91eee0cab208e512'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.419772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e0bbf31a4148cc8fa2264d0d0de4ac', 'request_id': 'req-29d67444-4e2f-46ca-8981-5dbfef82e10b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.420792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd61028a8804fba91eee0cab208e512'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.432752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e0bbf31a4148cc8fa2264d0d0de4ac', 'request_id': 'req-29d67444-4e2f-46ca-8981-5dbfef82e10b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.482756 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-49bd13fe-cdb1-480b-b865-99e7b7c3fb5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd61028a8804fba91eee0cab208e512'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.491397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1060/2088] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:18 2022] DELETE /identity/v3/users/afd61028a8804fba91eee0cab208e512 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:18.496018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:18.496546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:18.497029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:18.522813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:18.524782 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:18.562430 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e0bbf31a4148cc8fa2264d0d0de4ac', 'request_id': 'req-29d67444-4e2f-46ca-8981-5dbfef82e10b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.581646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] The token cache is being invalidate because project 56e0bbf31a4148cc8fa2264d0d0de4ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:18.582059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.582828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:18.597780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1029/2089] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:18 2022] DELETE /identity/v3/projects/56e0bbf31a4148cc8fa2264d0d0de4ac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:18.627040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1061/2090] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:18.640687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:18.640687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:18.730939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:18.760651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d67444-4e2f-46ca-8981-5dbfef82e10b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:18.872815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:18.873436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:18.873715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:18.873986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:18.886118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:18.888730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:18.911409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1030/2091] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:20.133122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:20.135957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:20.158913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553bdf7e-2d85-4a6a-aa97-1cb00eb401e7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:20.166740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:20.167406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:20.167687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:20.168003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] PATH_INFO: `/v3/projects/19fe796def664183a9867d58cf463d19` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:20.175614 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fe796def664183a9867d58cf463d19)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:20.177604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:20.337215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fe796def664183a9867d58cf463d19', 'request_id': 'req-56146206-c74d-47c1-8f61-81c135b97054'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.344054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fe796def664183a9867d58cf463d19', 'request_id': 'req-56146206-c74d-47c1-8f61-81c135b97054'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.419966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fe796def664183a9867d58cf463d19', 'request_id': 'req-56146206-c74d-47c1-8f61-81c135b97054'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.444165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] The token cache is being invalidate because project 19fe796def664183a9867d58cf463d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:20.444532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.445375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.450018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1062/2092] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:20 2022] DELETE /identity/v3/projects/19fe796def664183a9867d58cf463d19 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:20.451353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:20.468025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d18cd-4efe-4c3d-8d78-bfcdef7c661b service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:20.556428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:20.557490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:20.557922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:20.558354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] PATH_INFO: `/v3/users/19a3ff0c4e4e4e329a54d00479311eaa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:20.560097 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a3ff0c4e4e4e329a54d00479311eaa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:20.567044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:20.785112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a3ff0c4e4e4e329a54d00479311eaa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.785797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a3ff0c4e4e4e329a54d00479311eaa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.797426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a3ff0c4e4e4e329a54d00479311eaa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.825451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a3ff0c4e4e4e329a54d00479311eaa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.907478 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a3ff0c4e4e4e329a54d00479311eaa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:20.916205 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1031/2093] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:20 2022] DELETE /identity/v3/users/19a3ff0c4e4e4e329a54d00479311eaa => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:21.556983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:21.556983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:21.579541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146206-c74d-47c1-8f61-81c135b97054 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:21.689538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:21.706762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:21.707550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:21.711534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:21.711534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] PATH_INFO: `/v3/projects/33d9cf8e6dfe4c40a20ee88847d5da00` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:21.712555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d9cf8e6dfe4c40a20ee88847d5da00)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:21.717561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:21.892118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d9cf8e6dfe4c40a20ee88847d5da00', 'request_id': 'req-73bb4b56-615c-46eb-bb75-a31eb2c3073c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:21.907838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d9cf8e6dfe4c40a20ee88847d5da00', 'request_id': 'req-73bb4b56-615c-46eb-bb75-a31eb2c3073c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:21.974146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d9cf8e6dfe4c40a20ee88847d5da00', 'request_id': 'req-73bb4b56-615c-46eb-bb75-a31eb2c3073c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.011047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] The token cache is being invalidate because project 33d9cf8e6dfe4c40a20ee88847d5da00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:22.011346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.012424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.018203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1063/2094] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:21 2022] DELETE /identity/v3/projects/33d9cf8e6dfe4c40a20ee88847d5da00 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:22.019531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:22.023568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:22.054085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b49b-3edb-408b-b6ab-15292539c1c3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:22.092728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:22.182648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:22.183488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:22.183830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:22.184183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] PATH_INFO: `/v3/users/178e55e7c1bc4091a0635d36524c8a5c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:22.185848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=178e55e7c1bc4091a0635d36524c8a5c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:22.188605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:22.432836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178e55e7c1bc4091a0635d36524c8a5c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.434840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178e55e7c1bc4091a0635d36524c8a5c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.455273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178e55e7c1bc4091a0635d36524c8a5c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.490845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178e55e7c1bc4091a0635d36524c8a5c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.596787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f6992cb3-7a30-4bac-9391-8d5d1a18d94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178e55e7c1bc4091a0635d36524c8a5c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:22.607907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1032/2095] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:22 2022] DELETE /identity/v3/users/178e55e7c1bc4091a0635d36524c8a5c => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:23.356035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:23.360316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:23.402067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4b56-615c-46eb-bb75-a31eb2c3073c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:23.507163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:23.507944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:23.508287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:23.508730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] PATH_INFO: `/v3/projects/dad70140903445048f45d71faf746ffb` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:23.522460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad70140903445048f45d71faf746ffb)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:23.528255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:23.705037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad70140903445048f45d71faf746ffb', 'request_id': 'req-08b4afac-65a3-4e55-af0c-4731cb7e7916'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:23.714044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad70140903445048f45d71faf746ffb', 'request_id': 'req-08b4afac-65a3-4e55-af0c-4731cb7e7916'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:23.785264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad70140903445048f45d71faf746ffb', 'request_id': 'req-08b4afac-65a3-4e55-af0c-4731cb7e7916'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:23.801900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] The token cache is being invalidate because project dad70140903445048f45d71faf746ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:23.802547 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:23.806882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-08b4afac-65a3-4e55-af0c-4731cb7e7916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:23.811892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1064/2096] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:23 2022] DELETE /identity/v3/projects/dad70140903445048f45d71faf746ffb => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:23.813603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:23.814083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:23.814200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:23.844896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:23.846275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:23.919788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1033/2097] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:23.928096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:23.928096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:23.928096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:23.958742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:23.960524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:23.979898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1065/2098] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:23.984621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:23.996783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a619387-50e0-4282-b8a3-0df3aa5bcaf3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.001962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.002618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.002899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.003171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.006092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:39:24.006181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.007862 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.031464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1034/2099] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:39:24.036094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.050025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b908eb-f86f-4807-ba7d-8c8cf9ea5071 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.057153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.057790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.058071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.058346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.059034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.062311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.084755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:24.208858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1066/2100] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:24 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:39:24.210695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.229230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8620fa-7502-4fd2-a498-867cff977dee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.266190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:24.417634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.418408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.418893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.419311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.420207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.423461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95189c28-ff51-4837-8c3b-28ce1a7e4556 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.705881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1035/2101] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:24 2022] POST /identity/v3/users => generated 292 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:24.710969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.729063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01192d7c-616a-440f-b342-86d4ac134b5d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.752504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.753186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.753683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.755000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.755687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.755974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.756974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.758014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.760197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.785024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1067/2102] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:24.786162 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.798668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d77f-92dd-4a45-bd09-a25bb7690bfa admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.803242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.805842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.805842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.805842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.805842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.807398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.815253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] MFA Rules not processed for user `e7a6ae7333d2422ebc9f616d482e527a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:24.816636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:24.826845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1068/2103] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:24.831317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.845161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519cc59-dd5f-46b4-b379-870406dd6cad admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.865018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.865736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.866024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.866305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] PATH_INFO: `/v3/projects/4cb6b4edda314963acad175089204599/users/906899df35c14f9eb20415c513a8b599/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.887554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb6b4edda314963acad175089204599, user_id=906899df35c14f9eb20415c513a8b599, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.890558 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bcc34-1365-4cf6-a29b-d0b59194bf7e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.931555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1036/2104] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:39:24.935985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:24.937924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:24.952419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699593ea-fb0a-498d-9afe-49f0c93e79e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:24.957112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:24.957749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.958025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.958295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:24.958964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:24.960915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:24.981577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1069/2105] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:24 2022] PUT /identity/v3/projects/4cb6b4edda314963acad175089204599/users/906899df35c14f9eb20415c513a8b599/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:24.987681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:24.987973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:24.988259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.023743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:25.025631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:25.075001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1037/2106] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:24 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:39:25.076484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.122910 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1070/2107] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:39:25.143988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.146880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:25.220363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a392a7-3d91-4e73-af27-1b73ed28ff94 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.228086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.228726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.229075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.229363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.230177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.234561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.277672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500db065-3051-4b1e-be51-b9c7643d03f7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.351955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.352529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.352816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.353096 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.368507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.377411 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1038/2108] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:25 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:25.377550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.377641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.377722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:25.386813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1071/2109] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:39:25.387132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07a505-acc3-46d6-ac85-1b7886f91a0b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.392908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.393602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.393958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.394307 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.395242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.397568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.401246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:25.418548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1039/2110] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:39:25.420812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.439831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac87d6c-de6c-41cf-a26b-6bc96ebcca87 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.445680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.446572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.446935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.447294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] PATH_INFO: `/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f/users/a19083c9d88349a39f15ca600b326ebc/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.465440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b7e26e5e4d40e98f0d2b9bb483665f, user_id=a19083c9d88349a39f15ca600b326ebc, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.471043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.584509 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1072/2111] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:25 2022] PUT /identity/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f/users/a19083c9d88349a39f15ca600b326ebc/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:25.588330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.605000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a542be8-9993-4b05-9db3-b15025e0d9e5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.717350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.717350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.717350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.717350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.717716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.717716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32780c-0d25-411b-b020-c2333d4d1724 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.737360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1040/2112] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:25.741772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:25.755836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad59e33-f76d-484f-9c4a-887d33b1eade admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:25.760834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:25.761513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:25.761806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:25.762067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] PATH_INFO: `/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f/users/a19083c9d88349a39f15ca600b326ebc/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:25.764179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b7e26e5e4d40e98f0d2b9bb483665f, user_id=a19083c9d88349a39f15ca600b326ebc, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:25.767193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:25.782818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1073/2113] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:25 2022] PUT /identity/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f/users/a19083c9d88349a39f15ca600b326ebc/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:35.230784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040feba-de5b-4196-8e1e-93263c9597e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:35.231392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040feba-de5b-4196-8e1e-93263c9597e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:35.231496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040feba-de5b-4196-8e1e-93263c9597e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:35.317593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8040feba-de5b-4196-8e1e-93263c9597e6 None None] MFA Rules not processed for user `906899df35c14f9eb20415c513a8b599`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:35.319833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8040feba-de5b-4196-8e1e-93263c9597e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:35.493482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1041/2114] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:35.498247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:35.538623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:35.538913 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:35.539187 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:35.578833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] MFA Rules not processed for user `a19083c9d88349a39f15ca600b326ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:35.580871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:35.621643 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842468b-ec21-48e0-b81a-2a79e600a62e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:35.637651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:35.638309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:35.638613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:35.638886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:35.640287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:35.642233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:35.723818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1042/2115] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:35.739325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:35.741387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:35.765183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:35.770842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1074/2116] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:35 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:35.773773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:35.883457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136812a-4881-45d4-9fa0-49ac508637ca None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:35.900560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c562-2d80-446f-bbcd-40570582fe8f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:35.914836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:35.917802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:35.917802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:35.917802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:35.917802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:35.919339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.022110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:36.022702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:36.022981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:36.023264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:36.044192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1075/2117] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:35 2022] POST /identity/v3/users => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:36.045212 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:36.071450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d68e48-8d47-4ad6-b0cf-d738477f089b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:36.087378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:36.088378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:36.088739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:36.089125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:36.092145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:36.095750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.119230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1076/2118] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:36.127816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:36.146623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05a3be-d550-46b6-952f-6627955a380e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:36.152704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:36.153748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:36.154203 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:36.154661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] PATH_INFO: `/v3/projects/78d4191d568f4ad282525ea044e300e3/users/3734128364bf4309b1a85d8985cf7204/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:36.174703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:36.176343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.184484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d4191d568f4ad282525ea044e300e3, user_id=3734128364bf4309b1a85d8985cf7204, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:36.188322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.189680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1043/2119] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:36.331967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1077/2120] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:36 2022] PUT /identity/v3/projects/78d4191d568f4ad282525ea044e300e3/users/3734128364bf4309b1a85d8985cf7204/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:36.339953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:36.370583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79257afc-d84c-461c-8b4c-d50b8dff320e service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:36.512743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:36.553322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:36.555049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:36.557906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:36.558226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:36.563808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:36.570597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c87f9-d37b-4bc8-986d-9c86f4113538 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.585103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1044/2121] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:36.588836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:36.596979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:36.608361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f84b6-df44-4435-8686-60d777455dc4 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:36.631438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:36.632828 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:36.633649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:36.634184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] PATH_INFO: `/v3/projects/78d4191d568f4ad282525ea044e300e3/users/3734128364bf4309b1a85d8985cf7204/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:36.640244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d4191d568f4ad282525ea044e300e3, user_id=3734128364bf4309b1a85d8985cf7204, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:36.645089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74633bc2-564d-4fd4-89e9-004ae6d2d5d7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:36.659746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1078/2122] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:36 2022] PUT /identity/v3/projects/78d4191d568f4ad282525ea044e300e3/users/3734128364bf4309b1a85d8985cf7204/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:40.709280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:40.722672 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:40.723519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:40.723519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:40.723519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:40.724167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:40.765917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] MFA Rules not processed for user `eb7a8349031b4f87a4f477e4a4e6ba2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:40.767922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:40.779561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:40.781429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:40.890106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1045/2123] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:40.955882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1079/2124] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:40.981613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:40.983325 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:41.138814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce4679-a6ba-438d-babd-965d216419c3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:41.147419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:41.148346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:41.148811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:41.149242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:41.160677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:41.163700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:41.197157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1046/2125] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:39:42.004060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:42.006408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:42.168570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c02a6-6a49-4254-b837-17200b45b05f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:42.179536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:42.180185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:42.180472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:42.180748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:42.182880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:42.335950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:42.337710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887fb9c-83de-486e-a0ec-f9c18c3f495b service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:42.358089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1080/2126] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:43.741162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:43.752661 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7a8c-613d-42e1-b279-ff0fef62db26 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:43.757810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:43.758475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:43.758794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:43.759072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:43.761419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:43.786908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:43.789099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:43.807259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1047/2127] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:44.791968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:44.792465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:44.792668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:44.841990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] MFA Rules not processed for user `3734128364bf4309b1a85d8985cf7204`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:44.843367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:44.877327 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:44.878954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:44.880958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:44.944661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1081/2128] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:44.957940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:44.959318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:44.991095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ceab3-e6eb-4336-b910-1e402371fd41 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:45.022650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:45.023649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:45.024253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:45.024685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] PATH_INFO: `/v3/users/b69eb168c92343b5819d4795eb85dd7a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:45.026339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69eb168c92343b5819d4795eb85dd7a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:45.029176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:45.034019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:45.082451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f68af3-8c76-4558-9989-c242563adc16 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:45.086878 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:45.087712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:45.088152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:45.088639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:45.183971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:45.185439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d6621-271b-449b-8b01-09425f5a3234 service glance] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:45.223327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1082/2129] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:39:45.234556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69eb168c92343b5819d4795eb85dd7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:45.244016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69eb168c92343b5819d4795eb85dd7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:45.261857 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69eb168c92343b5819d4795eb85dd7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:45.287834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69eb168c92343b5819d4795eb85dd7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:45.356072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69eb168c92343b5819d4795eb85dd7a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:45.368280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1048/2130] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:39:44 2022] DELETE /identity/v3/users/b69eb168c92343b5819d4795eb85dd7a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:39:45.568244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:45.568533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:45.568806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:45.614350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:45.615896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:45.696752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1083/2131] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:45.719829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:45.721367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:45.753717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:45.753717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:45.790579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:45.830618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79e1a-793c-4d72-af37-0fff8712ee9d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:45.836915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:45.837592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:45.838020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:45.838234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:45.848810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:45.850594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147918a6-d195-4e90-9043-6de3027934b0 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:45.865457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1049/2132] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:39:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:39:45.872718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2d21-c4b8-4e71-8f2b-9266c4e93b7d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:45.880595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:45.881353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:45.881837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:45.882121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] PATH_INFO: `/v3/projects/62fb03abd1f64d748b132fa450f641f8` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:45.883482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62fb03abd1f64d748b132fa450f641f8)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:45.885417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:46.046770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62fb03abd1f64d748b132fa450f641f8', 'request_id': 'req-b9ebf325-5164-4263-adb2-966aefeb6e20'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:46.059041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62fb03abd1f64d748b132fa450f641f8', 'request_id': 'req-b9ebf325-5164-4263-adb2-966aefeb6e20'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:46.138887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62fb03abd1f64d748b132fa450f641f8', 'request_id': 'req-b9ebf325-5164-4263-adb2-966aefeb6e20'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:46.175226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] The token cache is being invalidate because project 62fb03abd1f64d748b132fa450f641f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:39:46.175552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:46.176284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b9ebf325-5164-4263-adb2-966aefeb6e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:39:46.178250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1084/2133] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:39:45 2022] DELETE /identity/v3/projects/62fb03abd1f64d748b132fa450f641f8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:46.183189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.184113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.184614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.221471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:46.222935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:46.457645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1050/2134] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:46.465389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.465682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.465962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.503058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:46.504165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:46.513804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1085/2135] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:46.518996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:46.531387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc3cf6-0ce8-46d5-9454-e2d9dc33f10f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:46.536764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:46.537769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.538160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.538650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.539511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:39:46.539809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:46.541492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:46.554262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:46.567067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:46.575519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1051/2136] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:39:46.581729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0c011-18f9-49c5-b51c-9a78cc53a59b None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:46.593403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:46.593403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.593403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.593403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.593903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:46.593903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:46.712888 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1086/2137] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:39:46 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:39:46.720372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:46.739950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b94467-a162-41cc-9082-6abfe037fb20 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:46.817231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:46.817863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.818125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.818395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.819093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:46.821222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:46.942839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1052/2138] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:39:46 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:39:46.953516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:46.969794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856bcad-f88d-4e14-91f9-edccf2bddaa3 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:46.975747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:46.981174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:46.981174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:46.981174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:46.981174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:46.984363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:46.996850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1087/2139] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 13:39:47.006784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:47.020572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dca79c-96bf-47d3-9619-22aa0bc118ab admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:47.034428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:47.035584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:47.036217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:47.036672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:47.037957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:47.040042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63408e76-b261-4dad-86ca-dfe717bdad04 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:47.062733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1053/2140] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:39:47.074037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:47.085767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4219763-d063-453e-aef9-fcd22f8a8adf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:47.094621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:47.095695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:47.096245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:47.096671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] PATH_INFO: `/v3/projects/b4b7c8d81c2349a38c027e9ecd0be712/users/593d2391fa0f480e92e5050efbdde308/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:47.121107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b7c8d81c2349a38c027e9ecd0be712, user_id=593d2391fa0f480e92e5050efbdde308, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:47.125279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:47.231701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1088/2141] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:39:47 2022] PUT /identity/v3/projects/b4b7c8d81c2349a38c027e9ecd0be712/users/593d2391fa0f480e92e5050efbdde308/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:39:47.233527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:47.233984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:47.234577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:47.287624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] MFA Rules not processed for user `593d2391fa0f480e92e5050efbdde308`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:47.289108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:47.394763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1054/2142] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:50.319993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:50.321766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:50.403052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:50.438106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37b2ba-6f70-489c-9bc0-025e4f9f0f86 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:50.444527 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:50.445287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:50.445576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:50.445860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:50.447734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:50.531159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:50.533213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e008d0-e6a7-4b19-9e1b-a5cabe6a94de service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:50.549872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1089/2143] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:51.649535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:51.651229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:51.753814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459381b6-01df-42d8-8335-1d62b6abf614 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:51.771185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:51.772219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:51.774484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:51.775035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:51.805516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:51.808070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:51.835934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1055/2144] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:58.218108 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:58.218712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:58.218810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:58.239760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] MFA Rules not processed for user `906899df35c14f9eb20415c513a8b599`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:39:58.240912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:58.309432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1090/2145] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:39:58.567039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:58.568440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:58.579728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:39:58.626596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d201-ec5f-420e-ab15-0ce77d89cfd5 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:58.630152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:58.630784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:58.631085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:58.631331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:58.633052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:58.643607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:58.645011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a734f57f-24f3-4b4f-be17-78ddf0f62b52 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:58.653102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1056/2146] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:39:59.008787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:39:59.010378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:59.082728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f22c9-ebc6-4996-a00a-7b42a6ce8ed0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:39:59.087731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:39:59.088343 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:39:59.088650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:39:59.088926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:39:59.090693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:39:59.101841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:39:59.103434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:39:59.113262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1091/2147] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:04.323963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99aea-dc68-4310-8e27-2b6fa3a05d30 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:04.324445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99aea-dc68-4310-8e27-2b6fa3a05d30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:04.324586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be99aea-dc68-4310-8e27-2b6fa3a05d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:04.344837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9be99aea-dc68-4310-8e27-2b6fa3a05d30 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:04.346060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9be99aea-dc68-4310-8e27-2b6fa3a05d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:04.408356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1057/2148] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:40:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:04.433951 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:04.436269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:04.476420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:04.566109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782c93d-78be-49c3-8743-0f98d3232719 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:04.572744 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:04.573410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:04.573866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:04.574149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:04.584903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:04.587206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:04.600204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1092/2149] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:40:14.401190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=47 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:14.593065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=24 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:15.370854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:15.371143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:15.371621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:15.404194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] MFA Rules not processed for user `e7a6ae7333d2422ebc9f616d482e527a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:15.405989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:15.478025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1058/2150] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:15.840239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:15.841710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:15.854609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c828aa1-2fac-4dd5-ab9c-f3ef04e0d588 service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:15.858933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:15.859544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:15.859835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:15.860123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:15.861892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:15.872013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:15.873521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:15.882461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1093/2151] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:40:25.065554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:25.075047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156473c0-0aa6-4f74-9398-585f5957d8d6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:25.079291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:25.079897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:25.080199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:25.080477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:25.082198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:25.093720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:25.095200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:25.104059 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1059/2152] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:25.107443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:25.117814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107bb29-126c-4ddb-bd1c-3e0824be5778 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:25.123691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:25.123691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:25.123691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:25.123691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:25.132207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:25.133659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8863b86-775c-4ad6-9998-378e9362a362 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:25.142884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1094/2153] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:40:26.015743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:26.019853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:26.035943 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082777-4015-4fa9-b780-9cb05996031e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:26.041912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:26.042933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:26.042933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:26.043138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:26.044866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:26.059639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:26.062741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:26.085611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1060/2154] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:26.322956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:26.323422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:26.323535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:26.345002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:26.346206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:26.353455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1095/2155] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:26.374956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:26.387044 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1eb3-a229-4751-93a6-769c59e2bd2c service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:26.392516 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:26.393142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:26.393423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:26.393699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:26.401664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:26.403274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:26.412994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1061/2156] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:40:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:40:36.339242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=15 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:36.411264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=43 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:37.179069 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:37.201144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0e1b6-d73d-42d0-a98d-cd8e6bf89161 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:37.207651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:37.208320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:37.208603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:37.208877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:37.210639 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:37.223865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:37.225795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf89b5c-d2ff-432b-995e-8dbb01d385a9 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:37.248896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1096/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:47.232891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:47.753082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:47.818005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54719-7ca5-400e-b1bf-86920dc9a96a service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:47.822633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:47.823267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:47.823612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:47.823823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] PATH_INFO: `/v3/users/593d2391fa0f480e92e5050efbdde308` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:47.825025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] RBAC: Authorizing `identity:delete_user(user_id=593d2391fa0f480e92e5050efbdde308)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:47.826993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:47.999778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '593d2391fa0f480e92e5050efbdde308'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.000760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '593d2391fa0f480e92e5050efbdde308'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.008802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '593d2391fa0f480e92e5050efbdde308'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.021623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '593d2391fa0f480e92e5050efbdde308'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.090467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '593d2391fa0f480e92e5050efbdde308'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.097232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1062/2158] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:40:47 2022] DELETE /identity/v3/users/593d2391fa0f480e92e5050efbdde308 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:40:48.101967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:48.102379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:48.102758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:48.122973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:48.123128 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:48.201667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1097/2159] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:48.214099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:48.216434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:48.285528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e0027-2800-45af-852d-93d94ea8808a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:48.289243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:48.289886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:48.290133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:48.290405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:48.299934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:48.301417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4767c9-7313-4e77-a26c-bfcdc876e348 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:48.311878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1063/2160] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:48.816399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:48.816399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:48.844588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c0391-dcaa-47d1-88eb-7969e2b038ce None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:48.858371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:48.859223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:48.859595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:48.859876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] PATH_INFO: `/v3/projects/b4b7c8d81c2349a38c027e9ecd0be712` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:48.863271 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b7c8d81c2349a38c027e9ecd0be712)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:48.865205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:48.973126 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b7c8d81c2349a38c027e9ecd0be712', 'request_id': 'req-9c60f3b1-aa5d-48f9-9249-ba9d22507953'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:48.979066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b7c8d81c2349a38c027e9ecd0be712', 'request_id': 'req-9c60f3b1-aa5d-48f9-9249-ba9d22507953'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:49.067697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b7c8d81c2349a38c027e9ecd0be712', 'request_id': 'req-9c60f3b1-aa5d-48f9-9249-ba9d22507953'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:49.079451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] The token cache is being invalidate because project b4b7c8d81c2349a38c027e9ecd0be712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:40:49.079743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:49.080383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9c60f3b1-aa5d-48f9-9249-ba9d22507953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:40:49.081863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1098/2161] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:40:48 2022] DELETE /identity/v3/projects/b4b7c8d81c2349a38c027e9ecd0be712 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:40:50.299695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.300201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.300575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.302281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1064/2162] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:40:50.308903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000010f3-2a18-4065-a5d1-c9c76d5bbb91 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.309404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000010f3-2a18-4065-a5d1-c9c76d5bbb91 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.309593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000010f3-2a18-4065-a5d1-c9c76d5bbb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.330177 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-000010f3-2a18-4065-a5d1-c9c76d5bbb91 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:50.331597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-000010f3-2a18-4065-a5d1-c9c76d5bbb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:50.414290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1099/2163] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:50.427465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:50.428528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:50.455726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:50.488667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf53fe-15d6-4cb6-b6af-4cc599810748 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:50.493873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:50.494535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.494828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.495104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.506999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:50.508426 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4ae0c-ec76-4a39-9d9a-2541f68ff9ce service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:50.519965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1065/2164] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:50.669497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.669994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.670437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.671940 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1100/2165] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:40:50.678670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c0eb7-1385-4d29-8054-cdc7d60f6cd3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.679016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c0eb7-1385-4d29-8054-cdc7d60f6cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.679302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c0eb7-1385-4d29-8054-cdc7d60f6cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.703360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-262c0eb7-1385-4d29-8054-cdc7d60f6cd3 None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:50.703360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-262c0eb7-1385-4d29-8054-cdc7d60f6cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:50.755318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1066/2166] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:50.767692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:50.769479 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:50.783205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3ed61-906e-48fc-88c3-27d01d1735cd None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:50.787186 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:50.787810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.788072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.788350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.795475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:50.796898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488bb01-48c8-424f-8d75-4bec76ca91c5 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:50.805418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1101/2167] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:40:50.919939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413799e-bd08-4b00-a04c-ed3b77a5ef2b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.920334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413799e-bd08-4b00-a04c-ed3b77a5ef2b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.920642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413799e-bd08-4b00-a04c-ed3b77a5ef2b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.922019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1067/2168] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:40:50.928658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.929037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.929378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:50.938282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:40:50.952399 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:40:50.953596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:40:50.963276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1102/2169] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:40:50.977985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413799e-bd08-4b00-a04c-ed3b77a5ef2b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:40:50.991506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413799e-bd08-4b00-a04c-ed3b77a5ef2b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:40:50.995305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:40:50.996063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:40:50.996377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:40:50.996655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:40:51.008110 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:40:51.009679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:40:51.021182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1068/2170] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:40:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:41:01.015823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=45 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:41:01.159457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:41:05.386760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:05.396938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7853a1a-1055-424b-8684-c0f5dc620fbf None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:05.400645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:05.401257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:05.401518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:05.401789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:05.403537 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:05.467092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:05.468504 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:05.478246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1103/2171] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:41:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:41:07.724835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:07.728150 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:07.746941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb68f98-1f2b-40b0-9388-07ecc7ee06ce service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:07.753072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:07.757987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:07.757987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:07.757987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:07.757987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:07.769573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:07.771466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007eed4-55c7-48e3-b9d1-e324dee39eae service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:07.783785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1069/2172] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:41:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:41:13.636237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:13.637667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:13.649418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc577c65-4fdb-4822-b417-b7de56258cec service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:13.653144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:13.653836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:13.654188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:13.654552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:13.656311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:13.719834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:13.721427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:13.731587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1104/2173] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:41:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:41:14.262134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:14.262134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:14.262134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:14.282476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:41:14.283826 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:14.343367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1070/2174] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:41:14.355630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:14.357028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:14.470730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a367a-75af-4037-aedf-c081b078ea16 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:14.474405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:14.475050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:14.475331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:14.475603 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:14.484448 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:14.487211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60856e2-c389-4abb-b7f5-ed86abad37b4 service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:14.495684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1105/2175] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:41:24.336625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=31 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:41:24.491971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:41:45.220023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:45.233393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24453ec0-da7f-46fe-99b3-ae196b822512 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:45.239539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:45.240146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:45.240428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:45.240703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:45.242427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:45.305320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:45.307020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:45.316525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1071/2176] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:41:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:41:49.750140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:49.758216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:49.758216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:49.769978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:41:49.771215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:41:49.781789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1106/2177] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:41:49.801605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:41:49.812778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb473d-51aa-48a3-9273-6743fc6ab7ec service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:41:49.817865 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:41:49.818492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:41:49.818831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:41:49.819101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:41:49.827029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:41:49.828598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:41:49.836611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1072/2178] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:41:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:41:59.764367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:41:59.833339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=34 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:06.405468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:06.406901 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:06.462330 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1fdc6-9247-47fc-bb03-d7e623a58c3d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:06.466797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:06.467392 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:06.467671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:06.467938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:06.469597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:06.526822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:06.528235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:06.537800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1107/2179] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:12.932639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:12.941834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d114d0-76c4-4b7d-bba1-ba5387b3d82c service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:12.945417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:12.946009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:12.946311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:12.946616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:12.948311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:13.013067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:13.014499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:13.024017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1073/2180] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:13.578775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:13.582542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:13.649014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5552387-708e-41b1-a963-335b5ceaac55 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:13.654218 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:13.654859 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:13.655144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:13.655428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:13.657152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:13.669015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:13.670615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:13.681450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1108/2181] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:13.705795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:13.707391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:13.721035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160702f-b78c-43ae-9e30-0cec7a4ae108 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:13.727534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:13.728288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:13.728653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:13.729085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] PATH_INFO: `/v3/users/eb7a8349031b4f87a4f477e4a4e6ba2b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:13.730285 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7a8349031b4f87a4f477e4a4e6ba2b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:13.732344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:13.912644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7a8349031b4f87a4f477e4a4e6ba2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:13.913295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7a8349031b4f87a4f477e4a4e6ba2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:13.920343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7a8349031b4f87a4f477e4a4e6ba2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:13.932622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7a8349031b4f87a4f477e4a4e6ba2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:13.995692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-302878e2-be54-422c-9d2c-3a48f9501cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7a8349031b4f87a4f477e4a4e6ba2b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.002171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1074/2182] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:42:13 2022] DELETE /identity/v3/users/eb7a8349031b4f87a4f477e4a4e6ba2b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:14.233276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.247805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60202eb3-ebc6-4052-bf00-c5dd8ef17405 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.292978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:14.308378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.309012 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.310223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.310223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] PATH_INFO: `/v3/projects/c20b3147c0d14452b9c5f22801c89990` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.310741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20b3147c0d14452b9c5f22801c89990)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:14.312684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:14.416151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20b3147c0d14452b9c5f22801c89990', 'request_id': 'req-9499361e-5db4-41a4-8494-662855323f2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.421956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20b3147c0d14452b9c5f22801c89990', 'request_id': 'req-9499361e-5db4-41a4-8494-662855323f2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.499109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20b3147c0d14452b9c5f22801c89990', 'request_id': 'req-9499361e-5db4-41a4-8494-662855323f2c'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.511669 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] The token cache is being invalidate because project c20b3147c0d14452b9c5f22801c89990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:42:14.511960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.512594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9499361e-5db4-41a4-8494-662855323f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:14.514079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1109/2183] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:42:14 2022] DELETE /identity/v3/projects/c20b3147c0d14452b9c5f22801c89990 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:14.519117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.519427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.519707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.540072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:14.541278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:14.600064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1075/2184] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:14.607697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.608174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.608279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.628157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:14.629341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:14.637684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1110/2185] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:14.641790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.651353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfaaff-be8e-4689-827f-b07722aeee15 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.656020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.656617 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.656900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.657173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.657847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:42:14.658152 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:14.659778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:14.661218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:14.671795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1076/2186] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:42:14.676302 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.686040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde308c4-6f3b-44e4-a541-101afac1b9ec None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.690745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.691333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.691610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.691879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.692502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:14.695142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:14.775393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1111/2187] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:42:14 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:42:14.779472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.789959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02b367-3b07-4ea6-ac85-383cb91ccbae admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.845842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.846442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.846842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.847120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.847785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:14.849691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:14.946224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1077/2188] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:42:14 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:42:14.950287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.960709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afece7f-4351-4c2b-b485-140837455922 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.965429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.966027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.966309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.966615 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:14.967256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:14.968854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:14.979319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1112/2189] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:42:14.983022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:14.993627 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674b78-7f14-473c-a3e3-d121cbf96362 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:14.998269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:14.998890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:14.999167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:14.999446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:15.000103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:15.001671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad64a4-9a2c-4fd0-8ff7-1a66891832d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:15.012846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1078/2190] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:42:15.017031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:15.027116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdaddd-a550-4185-9dc2-57798428718b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:15.031658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:15.032299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:15.032582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:15.032858 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] PATH_INFO: `/v3/projects/8c2ebd079e654ab3872c60a5963819b7/users/f617e6c65dfd4dfbbe829c1613ea24c4/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:15.047538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2ebd079e654ab3872c60a5963819b7, user_id=f617e6c65dfd4dfbbe829c1613ea24c4, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:15.050434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:15.160826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1113/2191] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:42:15 2022] PUT /identity/v3/projects/8c2ebd079e654ab3872c60a5963819b7/users/f617e6c65dfd4dfbbe829c1613ea24c4/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:15.171172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fe193-b491-45ec-ab9d-fe75416848d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:15.174320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fe193-b491-45ec-ab9d-fe75416848d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:15.174320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fe193-b491-45ec-ab9d-fe75416848d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:15.200279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f78fe193-b491-45ec-ab9d-fe75416848d9 None None] MFA Rules not processed for user `f617e6c65dfd4dfbbe829c1613ea24c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:15.201465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f78fe193-b491-45ec-ab9d-fe75416848d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:15.265681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1079/2192] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:15.276321 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:15.278075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:15.336116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54753cc9-6068-4b09-b6dd-3af4b466b2e2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:15.339667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:15.340264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:15.340538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:15.340806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:15.348801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:15.350849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:15.359355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1114/2193] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:25.262866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:25.355512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:37.857677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f62f-9445-4c2e-8c09-886d6b6602df None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:37.858251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f62f-9445-4c2e-8c09-886d6b6602df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:37.858251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f62f-9445-4c2e-8c09-886d6b6602df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:37.877277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0625f62f-9445-4c2e-8c09-886d6b6602df None None] MFA Rules not processed for user `020ff888a8a44d2eb6961829407637a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:37.878488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0625f62f-9445-4c2e-8c09-886d6b6602df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:37.933694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1080/2194] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:42.074689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:42.135294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f55ff-2606-4862-9a64-47a517627d79 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:42.139754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:42.140370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:42.140652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:42.140925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] PATH_INFO: `/v3/users/f617e6c65dfd4dfbbe829c1613ea24c4` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:42.142031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f617e6c65dfd4dfbbe829c1613ea24c4)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:42.143971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:42.324182 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f617e6c65dfd4dfbbe829c1613ea24c4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:42.331002 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f617e6c65dfd4dfbbe829c1613ea24c4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:42.343606 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f617e6c65dfd4dfbbe829c1613ea24c4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:42.344276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f617e6c65dfd4dfbbe829c1613ea24c4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:42.395751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f617e6c65dfd4dfbbe829c1613ea24c4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:42.402265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1115/2195] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:42:42 2022] DELETE /identity/v3/users/f617e6c65dfd4dfbbe829c1613ea24c4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:42.406780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:42.408919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:42.408919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:42.427215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:42.428413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:42.487766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1081/2196] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:42.497648 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:42.499319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:42.556789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da59c-1da8-465e-b18d-1c4f5ddd4e33 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:42.560592 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:42.561192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:42.561468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:42.561738 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:42.570471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:42.571914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94446d5d-3083-4d7b-95f4-f196e8f5d3dc service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:42.581692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1116/2197] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:42.975178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:42.976595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:43.001552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a54fa-7d68-436f-b3ff-c3a18f1fe5c6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.003863 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.004512 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.004787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.005064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] PATH_INFO: `/v3/projects/8c2ebd079e654ab3872c60a5963819b7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.006304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2ebd079e654ab3872c60a5963819b7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.008802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.112207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2ebd079e654ab3872c60a5963819b7', 'request_id': 'req-ffc2fec7-1d2a-45f1-9934-db025e30a040'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:43.118436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2ebd079e654ab3872c60a5963819b7', 'request_id': 'req-ffc2fec7-1d2a-45f1-9934-db025e30a040'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:43.173824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2ebd079e654ab3872c60a5963819b7', 'request_id': 'req-ffc2fec7-1d2a-45f1-9934-db025e30a040'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:43.186962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] The token cache is being invalidate because project 8c2ebd079e654ab3872c60a5963819b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:42:43.187248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:43.190367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffc2fec7-1d2a-45f1-9934-db025e30a040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:43.190367 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1082/2198] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:42:42 2022] DELETE /identity/v3/projects/8c2ebd079e654ab3872c60a5963819b7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:43.194485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.194974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.195085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.215678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:43.216890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:43.233950 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:43.281522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1117/2199] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:43.289422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.289714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.290050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.311174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:43.312520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:43.320732 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1083/2200] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:43.325581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.335507 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cf90b-0f0b-48a5-b62e-f0161e677123 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.340141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.340740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.341021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.341293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.341965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:42:43.342299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.343956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.359942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1118/2201] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:42:43.361280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.371021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1b69f-01e5-4a6b-9e30-8013734f482e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.376018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.376622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.376925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.377197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.377828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.379740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.453604 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:43.463509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1084/2202] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:42:43 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:42:43.465025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.475423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635841e-972a-42d1-8af2-4c234c92668d admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.532778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.533386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.533663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.533937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.534590 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.536498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.613739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1119/2203] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:42:43 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:42:43.617778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.629008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51f3f4-c5bc-4162-bade-500bff4b3dfe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.633680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.635886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.635886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.635886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.635886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.637261 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.647849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1085/2204] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:42:43.651874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.662326 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b912a-4926-4653-ae35-ddf566dccb06 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.668190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.668795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.669078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.669404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.670044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.671664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89279da-a710-4634-a82c-c3c4d606ef98 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.683626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1120/2205] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:42:43.687845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.698304 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62881be-4d60-48a1-a8be-d7dd2262676b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.703106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.703796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.704080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.704357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] PATH_INFO: `/v3/projects/e6193da6649c4e37a0a813f75fd2ce46/users/d8b09cd889f6441380fcb29305906c83/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.720015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6193da6649c4e37a0a813f75fd2ce46, user_id=d8b09cd889f6441380fcb29305906c83, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.722822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.785482 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1086/2206] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:42:43 2022] PUT /identity/v3/projects/e6193da6649c4e37a0a813f75fd2ce46/users/d8b09cd889f6441380fcb29305906c83/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:43.796363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.797710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.797710 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.826889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] MFA Rules not processed for user `d8b09cd889f6441380fcb29305906c83`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:43.828131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:43.891600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1121/2207] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:43.903606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:43.905351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:43.963212 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c98257-c4ec-4ff5-b4e8-02e981b923ce admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:43.967429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:43.968030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:43.968305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:43.968573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:43.976881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:43.978296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cebc5-000e-4d0b-b600-e2e7b4ae4f63 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:43.987800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1087/2208] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:53.888565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:53.983683 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:54.941766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:55.045868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0207559-9fa0-4201-8476-db4e5f2b21a6 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:55.050722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:55.051347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:55.051628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:55.051900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] PATH_INFO: `/v3/users/d8b09cd889f6441380fcb29305906c83` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:55.053025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b09cd889f6441380fcb29305906c83)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:55.054981 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:55.222766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b09cd889f6441380fcb29305906c83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:55.230249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b09cd889f6441380fcb29305906c83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:55.242137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b09cd889f6441380fcb29305906c83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:55.242825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b09cd889f6441380fcb29305906c83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:55.304349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b09cd889f6441380fcb29305906c83'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:55.310247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1122/2209] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:42:54 2022] DELETE /identity/v3/users/d8b09cd889f6441380fcb29305906c83 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:42:55.314709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:55.315173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:55.315287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:55.364300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:42:55.365479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:55.423331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1088/2210] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:42:55.432930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:55.434314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:55.475821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:42:55.507178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906aa1c-6bc7-41dc-87bf-c50b3f99e9a5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:55.511374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:55.511972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:55.512252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:55.512536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:55.521988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:55.523438 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:55.533406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1123/2211] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:42:55.911657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:42:55.913143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:42:55.928836 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7317d2-9d78-42b6-bfa3-771d034e43bd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:42:55.935030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:42:55.935669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:42:55.935947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:42:55.936200 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] PATH_INFO: `/v3/projects/e6193da6649c4e37a0a813f75fd2ce46` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:42:55.937324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6193da6649c4e37a0a813f75fd2ce46)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:42:55.939429 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:42:56.041203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6193da6649c4e37a0a813f75fd2ce46', 'request_id': 'req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:56.046856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6193da6649c4e37a0a813f75fd2ce46', 'request_id': 'req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:56.121410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6193da6649c4e37a0a813f75fd2ce46', 'request_id': 'req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:56.132951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] The token cache is being invalidate because project e6193da6649c4e37a0a813f75fd2ce46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:42:56.133236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:56.133858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dbf6b170-dd78-45e9-8bbf-66b4f9d3382d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:42:56.136976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1089/2212] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:42:55 2022] DELETE /identity/v3/projects/e6193da6649c4e37a0a813f75fd2ce46 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:43:05.534600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=16 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:43:06.129975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:43:44.009836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:43:44.011468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:43:44.108205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292662d5-41bd-4b56-a7a4-57a691f2ad6e service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:43:44.170476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:43:44.171380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:43:44.171662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:43:44.171939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:43:44.173757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:43:44.185584 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:43:44.187344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:43:44.205389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1124/2213] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:43:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:43:49.167003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:43:49.167514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:43:49.167623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:43:49.187631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] MFA Rules not processed for user `6c828726ae814c0f879a5aa69f647a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:43:49.190825 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:43:49.242609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1090/2214] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:43:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:43:54.205375 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=67 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:43:59.236154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=25 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:20.511976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:20.513393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:20.588242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7dcb4-2805-47e9-830a-282871139939 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:20.591825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:20.592417 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:20.592695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:20.592967 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:20.595296 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:20.654560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:20.655990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:20.666256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1125/2215] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:24.455020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:24.456495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:24.471762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f104308-e620-42e7-9b40-704b950dc607 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:24.476495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:24.476996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:24.477277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:24.477730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] PATH_INFO: `/v3/users/020ff888a8a44d2eb6961829407637a9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:24.478887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=020ff888a8a44d2eb6961829407637a9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:24.480805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:24.668461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020ff888a8a44d2eb6961829407637a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:24.669139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020ff888a8a44d2eb6961829407637a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:24.676941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020ff888a8a44d2eb6961829407637a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:24.689135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020ff888a8a44d2eb6961829407637a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:24.753386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-16ad30ce-d69f-4f98-83db-fc35938bb757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020ff888a8a44d2eb6961829407637a9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:24.759736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1091/2216] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:44:24 2022] DELETE /identity/v3/users/020ff888a8a44d2eb6961829407637a9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:44:25.010729 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.020946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7bb09-147a-4b5a-93ff-1fd8276a7e81 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.078443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.079099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.079379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.079655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] PATH_INFO: `/v3/projects/a3e2936fe3344e60b6f3dc918b9c738e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.081244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e2936fe3344e60b6f3dc918b9c738e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.083171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.164962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:25.176865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e2936fe3344e60b6f3dc918b9c738e', 'request_id': 'req-543e51ff-3e94-4e5f-befa-bb49499f4c55'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:25.182366 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e2936fe3344e60b6f3dc918b9c738e', 'request_id': 'req-543e51ff-3e94-4e5f-befa-bb49499f4c55'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:25.245051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e2936fe3344e60b6f3dc918b9c738e', 'request_id': 'req-543e51ff-3e94-4e5f-befa-bb49499f4c55'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:25.260942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] The token cache is being invalidate because project a3e2936fe3344e60b6f3dc918b9c738e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:44:25.260942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:25.260942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-543e51ff-3e94-4e5f-befa-bb49499f4c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:44:25.260942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1126/2217] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:44:25 2022] DELETE /identity/v3/projects/a3e2936fe3344e60b6f3dc918b9c738e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:44:25.265219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.265507 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.265782 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.314725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:25.316008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:25.374527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1092/2218] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:25.383187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.387228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.387228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.404089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:25.405294 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:25.413890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1127/2219] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:25.418174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.428261 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa78a20-4795-431b-aa47-c12e155f1929 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.432851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.437002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.448534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1093/2220] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:44:25.452709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.462697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf783c3-4882-4262-b517-caa7bbed4fc3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.467525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.468134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.468416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.468688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.469340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.471580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.561322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1128/2221] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:44:25 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:44:25.565845 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.576153 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db217b9-18b3-424f-9861-adadb6354166 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.609318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:25.635622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.636232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.636510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.636781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.637404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.639336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.711992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1094/2222] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:44:25 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:44:25.716092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.727180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4220f9-035a-4880-968c-f663c9be4f20 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.731869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.732468 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.732748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.733024 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.733668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.735306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.746084 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1129/2223] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:44:25.750116 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.768302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1ef1-9d53-460a-a96f-54913f0f1ab7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.773419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.774028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.774310 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.774616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.775263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.776847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8d14c-bd38-454e-a697-697ffecc2f59 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.787527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1095/2224] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:44:25.791610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:25.801812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98197528-e03c-4e29-ad0a-2154fab81be5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:25.810311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:25.810311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.810311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.810311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] PATH_INFO: `/v3/projects/723dfd3a60ee412b8569bb0a6081dc7a/users/c996056bd0354124ae74e79abf2753a8/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.822180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=723dfd3a60ee412b8569bb0a6081dc7a, user_id=c996056bd0354124ae74e79abf2753a8, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:25.824841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:25.923224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1130/2225] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:44:25 2022] PUT /identity/v3/projects/723dfd3a60ee412b8569bb0a6081dc7a/users/c996056bd0354124ae74e79abf2753a8/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:44:25.928558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5068-a48b-4557-b65b-061baea0ab66 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:25.928877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5068-a48b-4557-b65b-061baea0ab66 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:25.929154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a5068-a48b-4557-b65b-061baea0ab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:25.948299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-698a5068-a48b-4557-b65b-061baea0ab66 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:25.949477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-698a5068-a48b-4557-b65b-061baea0ab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:26.008262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1096/2226] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:44:26.018362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:26.020166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:26.077536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f1c1-1317-4ec2-9e2f-59bdfc13f7cf admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:26.081722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:26.082352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:26.082673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:26.082949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:26.091814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:26.093225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683b6f7-16b3-4a93-b70a-7428a45192a4 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:26.103133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1131/2227] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:31.469156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:31.469730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:31.469730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:31.502385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] MFA Rules not processed for user `c996056bd0354124ae74e79abf2753a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:31.503647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:31.585975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1097/2228] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:31.592418 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0061-7ec4-41a8-b3e3-c792d6a04df8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:31.593092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0061-7ec4-41a8-b3e3-c792d6a04df8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:31.593378 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0061-7ec4-41a8-b3e3-c792d6a04df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:31.600405 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:31.602751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:31.616524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-918e0061-7ec4-41a8-b3e3-c792d6a04df8 None None] MFA Rules not processed for user `e7a6ae7333d2422ebc9f616d482e527a`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:31.617945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-918e0061-7ec4-41a8-b3e3-c792d6a04df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:31.662776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:31.684579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1132/2229] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:31.725179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f65bdf-26c9-404d-a1cf-f9b290a6c1e8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:31.735469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:31.736682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:31.737009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:31.737287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:31.747156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:31.748796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:31.758410 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1098/2230] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:44:34.099470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:34.099955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:34.100057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:34.108593 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:34.123599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] MFA Rules not processed for user `c996056bd0354124ae74e79abf2753a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:34.124889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:34.136914 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1133/2231] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:35.536555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:35.538028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:35.622238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00537a-15a0-4378-a76b-def53210fe56 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:35.626034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:35.626678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:35.626961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:35.627678 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:35.629066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:35.645989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:35.645989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:35.653320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1099/2232] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:43.223235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:43.232781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02c049-e610-4049-8ed4-32ce7adb5273 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:43.237284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:43.237882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:43.238163 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:43.238437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:43.240148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:43.253989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:43.255471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:43.266133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1134/2233] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:43.763218 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:43.772306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520e2a-876c-413d-b360-95c8b047133f service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:43.775910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:43.776500 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:43.776780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:43.777055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:43.778896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:43.793119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:43.800223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610c88c-838a-4c0e-9d62-4ec7f0e32849 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:43.806277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1100/2234] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:45.087599 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:45.088991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:45.100848 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0cedd-79f5-4dbb-827e-8927621e65b9 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:45.104383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:45.104973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:45.105253 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:45.105524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:45.107189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:45.164907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:45.166332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:45.175969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1135/2235] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:44:53.713423 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:53.714422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:53.715144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:53.746391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:44:53.747540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:53.804934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1101/2236] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:44:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:44:53.933667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:53.935371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:54.026731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f39254-8c60-468d-a41c-fc792dc72185 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:54.031771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:54.032450 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:54.032736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:54.033008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:54.042881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:54.044725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:54.053415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1136/2237] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:44:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 13:44:55.158761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:55.168522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ca96a-f400-46a8-9c6b-84e2f713a022 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:55.172422 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:55.173122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:55.173466 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:55.173808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:55.176290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:55.188726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:44:55.200242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:55.201694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:55.227956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1102/2238] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:44:58.898688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:44:58.900249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:58.974496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b383a2-b397-4396-8c74-37519bb9f5ad service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:44:58.979652 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:44:58.980265 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:44:58.980546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:44:58.980849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:44:58.982595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:44:58.994742 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:44:58.996285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:44:59.007083 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1137/2239] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:44:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:02.793311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:02.795158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:02.808412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226baa0a-8f24-45ad-882d-422c1c2bcf5a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:02.813263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:02.813833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:02.814115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:02.814393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] PATH_INFO: `/v3/users/906899df35c14f9eb20415c513a8b599` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:02.815573 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=906899df35c14f9eb20415c513a8b599)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:02.818191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:02.998843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906899df35c14f9eb20415c513a8b599'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:02.999515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906899df35c14f9eb20415c513a8b599'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.006860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906899df35c14f9eb20415c513a8b599'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.019037 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906899df35c14f9eb20415c513a8b599'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.087165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906899df35c14f9eb20415c513a8b599'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.093184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1103/2240] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:45:02 2022] DELETE /identity/v3/users/906899df35c14f9eb20415c513a8b599 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:03.331868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:03.337320 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:03.343041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bc34b-fe88-44b4-8f2c-d269942e9327 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:03.398899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:03.399523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:03.399801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:03.400070 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] PATH_INFO: `/v3/projects/4cb6b4edda314963acad175089204599` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:03.401161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb6b4edda314963acad175089204599)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:03.403087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:03.507484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb6b4edda314963acad175089204599', 'request_id': 'req-680d9325-0ef0-49cd-8043-15fea0106d96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.513008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb6b4edda314963acad175089204599', 'request_id': 'req-680d9325-0ef0-49cd-8043-15fea0106d96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.586908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb6b4edda314963acad175089204599', 'request_id': 'req-680d9325-0ef0-49cd-8043-15fea0106d96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.598226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] The token cache is being invalidate because project 4cb6b4edda314963acad175089204599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:45:03.598457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.599100 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.600567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1138/2241] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:45:03 2022] DELETE /identity/v3/projects/4cb6b4edda314963acad175089204599 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:03.604539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:03.605978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:03.618917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c3c77-1d55-435f-9161-059a3c015f48 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:03.674476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:03.675198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:03.675510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:03.675784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] PATH_INFO: `/v3/users/3734128364bf4309b1a85d8985cf7204` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:03.676951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3734128364bf4309b1a85d8985cf7204)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:03.678909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:03.860580 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3734128364bf4309b1a85d8985cf7204'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.861215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3734128364bf4309b1a85d8985cf7204'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.868295 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3734128364bf4309b1a85d8985cf7204'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.879551 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:03.881472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3734128364bf4309b1a85d8985cf7204'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.950362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-650feb26-99d6-4720-b5fa-5406d95dbf98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3734128364bf4309b1a85d8985cf7204'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:03.957412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1104/2242] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:45:03 2022] DELETE /identity/v3/users/3734128364bf4309b1a85d8985cf7204 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:04.207869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:04.209308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:04.222369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d9325-0ef0-49cd-8043-15fea0106d96 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:04.279847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:04.280485 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.280769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.281047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] PATH_INFO: `/v3/projects/78d4191d568f4ad282525ea044e300e3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.282266 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d4191d568f4ad282525ea044e300e3)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:04.284407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:04.387255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:04.396556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d4191d568f4ad282525ea044e300e3', 'request_id': 'req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:04.401977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d4191d568f4ad282525ea044e300e3', 'request_id': 'req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:04.464051 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d4191d568f4ad282525ea044e300e3', 'request_id': 'req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:04.475671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] The token cache is being invalidate because project 78d4191d568f4ad282525ea044e300e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:45:04.487199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:04.487199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-1ec24f61-8b6a-40b9-9c3c-ed2186ee846b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:45:04.487199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1139/2243] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:45:04 2022] DELETE /identity/v3/projects/78d4191d568f4ad282525ea044e300e3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:04.488138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.488138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.488138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.504079 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:04.505271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:04.565597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1105/2244] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:04.573335 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.573627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.573955 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.593546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:04.594767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:04.603156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1140/2245] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:04.607402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:04.617607 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042be8c-9c75-4c53-93ec-f0ed59833095 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:04.622170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:04.622793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.623071 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.623340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.623981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:45:04.624280 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:04.625853 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:04.637189 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1106/2246] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:45:04.641470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:04.650819 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723fb8c-6261-433b-8417-43074ff5df29 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:04.655553 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:04.656149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.656432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.656705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.657340 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:04.659808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:04.788877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1141/2247] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:45:04 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:45:04.792749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:04.803039 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc849627-d5ef-4c2b-951e-0acb82d8e24c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:04.858851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:04.859448 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.859729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.860004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.860641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:04.862676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:04.956842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1107/2248] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:45:04 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:45:04.960989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:04.971365 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11140cb7-4000-4133-baf0-ab7bb447c6ca admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:04.977104 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:04.977737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:04.977971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:04.980758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:04.980758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:04.980758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:04.990401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1142/2249] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:45:04.994351 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:05.007245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3ba6f-d5b3-47c9-885b-380d14424266 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:05.011978 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:05.012614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:05.012897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:05.013182 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:05.013894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:05.015519 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817dd60-9c09-43c3-a3b3-ddc7afda2dae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:05.026840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1108/2250] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:45:05.031041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:05.041724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb141-419c-49a6-a8d2-3d61ee596355 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:05.046576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:05.047219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:05.047500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:05.047774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] PATH_INFO: `/v3/projects/76384f17c9384958983b188d3617be7e/users/8f759f41b4d44a53a0b60443b4df0a75/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:05.062273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76384f17c9384958983b188d3617be7e, user_id=8f759f41b4d44a53a0b60443b4df0a75, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:05.064943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:05.123674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1143/2251] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:45:05 2022] PUT /identity/v3/projects/76384f17c9384958983b188d3617be7e/users/8f759f41b4d44a53a0b60443b4df0a75/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:05.129222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0ff25-a674-4735-a64b-109a2de2bd5c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:05.129505 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0ff25-a674-4735-a64b-109a2de2bd5c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:05.129780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0ff25-a674-4735-a64b-109a2de2bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:05.149343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71f0ff25-a674-4735-a64b-109a2de2bd5c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:05.150742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71f0ff25-a674-4735-a64b-109a2de2bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:05.197891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:05.210433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1109/2252] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:05.220071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:05.221750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:05.279195 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d64b74-6045-4b62-a7b5-98fc8aab96b8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:05.282936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:05.283533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:05.283922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:05.284198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:05.293235 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:05.294902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:05.304115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1144/2253] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:45:10.341403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:10.342421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:10.342897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:10.374770 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] MFA Rules not processed for user `8f759f41b4d44a53a0b60443b4df0a75`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:10.376095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:10.458030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1110/2254] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:10.461997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:10.473662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fb3e0-7e57-47fc-a9a2-e7c1cf8905f6 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:10.479202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:10.479822 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:10.480113 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:10.480387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:10.481039 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:10.483054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:10.583748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1145/2255] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:45:10 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:45:10.587693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:10.599880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44598f64-917b-4a77-a0fc-d62d01504a12 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:10.661131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:10.661875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:10.662221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:10.662613 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:10.663555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:10.665257 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:10.833993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1111/2256] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:45:10 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:45:10.835800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:10.847965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605bcae-ecfd-4ac8-993d-19d9616006d2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:10.853015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:10.853631 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:10.853915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:10.854183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:10.854898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:10.856543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:10.868540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1146/2257] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:45:10.872807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:10.884057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2b24-2670-43b4-8ebd-46b36e667045 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:10.889199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:10.889878 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:10.890557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:10.890557 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] PATH_INFO: `/v3/projects/87f91edec8244c70ab273793676adfbf/users/214b9e38cc194a98ae7767679e852761/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:10.909958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f91edec8244c70ab273793676adfbf, user_id=214b9e38cc194a98ae7767679e852761, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:10.912759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:10.979818 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1112/2258] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:45:10 2022] PUT /identity/v3/projects/87f91edec8244c70ab273793676adfbf/users/214b9e38cc194a98ae7767679e852761/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:10.983755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:10.993758 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e408-dcb9-40ea-bf2e-09e822362569 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:11.016632 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:11.056903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:11.057511 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:11.057791 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:11.058062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:11.058763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:11.060367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce54d3-71e4-45b8-8a94-94869f5f36d7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:11.071246 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1147/2259] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:45:11.075553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:11.086668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15ef8f-3934-4296-b2ba-59cb3fda6f82 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:11.091379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:11.092029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:11.092311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:11.092588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] PATH_INFO: `/v3/projects/87f91edec8244c70ab273793676adfbf/users/214b9e38cc194a98ae7767679e852761/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:11.094499 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f91edec8244c70ab273793676adfbf, user_id=214b9e38cc194a98ae7767679e852761, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:11.097206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:11.107494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1113/2260] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:45:11 2022] PUT /identity/v3/projects/87f91edec8244c70ab273793676adfbf/users/214b9e38cc194a98ae7767679e852761/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:45:16.335924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bee623-fe51-4e00-b8da-ee83f9e439ee None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:16.336404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bee623-fe51-4e00-b8da-ee83f9e439ee None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:16.336502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bee623-fe51-4e00-b8da-ee83f9e439ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:16.370655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-24bee623-fe51-4e00-b8da-ee83f9e439ee None None] MFA Rules not processed for user `214b9e38cc194a98ae7767679e852761`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:16.372129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-24bee623-fe51-4e00-b8da-ee83f9e439ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:16.457442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1148/2261] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:16.475201 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:16.475694 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:16.540210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:16.583579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a024c1-c485-4c18-b57c-9d299fda874e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:16.589253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:16.590011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:16.590299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:16.590605 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:16.659032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:16.660776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:16.672574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1114/2262] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:19.255989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:19.256486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:19.257251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:19.277133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] MFA Rules not processed for user `8f759f41b4d44a53a0b60443b4df0a75`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:45:19.278344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:19.286168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1149/2263] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:45:26.668337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=40 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:29.271484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=57 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:35.004389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:35.005721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:35.020501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f2a9-c7b9-400b-885b-8e5ff2de2274 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:35.026165 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:35.026911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:35.027193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:35.027468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:35.029166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:35.109464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:35.111118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:35.130420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1115/2264] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:43.503032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:43.504445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:43.562894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad768f4-517d-487b-95c9-58d85b232f96 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:43.566532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:43.575118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:43.575118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:43.575118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:43.575118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:43.579833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:43.581259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:43.589531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1150/2265] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:44.358602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:44.362907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:44.430710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c768a-e86a-41ba-bf81-a91037c75661 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:44.435664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:44.436278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:44.436558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:44.436834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:44.438612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:44.449854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:44.451481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:44.461274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1116/2266] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:47.681291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:47.692191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413dbf5-dc3f-4fd1-90d3-7f27f092ed73 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:47.696472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:47.697145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:47.697559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:47.697899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:47.699795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:47.710159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:47.714692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:47.722722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1151/2267] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:45:48.587487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:45:48.600653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a751868-ff1c-4276-8865-c0b821ae7b05 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:45:48.607027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:45:48.607639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:45:48.607915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:45:48.608184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:45:48.609906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:45:48.622955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:45:48.624624 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb6012-1c4e-43ff-b3c7-df5cde4b377f service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:45:48.637912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1117/2268] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:45:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:45:57.718454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=32 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:45:58.630781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=79 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:02.588544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:02.603139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e046b5e-1f93-41a2-b7d5-442727acb7fb service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:02.608671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:02.609281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:02.609568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:02.609846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:02.611725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:02.624332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:02.629718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:02.637711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1152/2269] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:46:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:46:04.610299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:04.610855 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:04.611125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:04.662051 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:04.663321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:04.732653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1118/2270] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:46:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:46:04.879168 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:04.880883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:04.974918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546520f6-2c66-4c21-b02d-55e9e9b76178 service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:04.980781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:04.981402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:04.981686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:04.981963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:04.991889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:04.994091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dee990-b385-4d01-a8e4-afd8656ae25d service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:05.003250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1153/2271] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:46:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:46:08.262959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:08.274935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7c0df-0f83-4494-8d1e-205d18f2a0ad None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:08.278993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:08.279787 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:08.280081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:08.280355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:08.291680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:08.293353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31489b9b-9a34-4ce1-ba03-e4a044748225 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:08.304062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1119/2272] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:46:14.999552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=47 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:18.299365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=46 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:24.401118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.401580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.401712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.403073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1154/2273] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:46:24.409047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e3a88-81ca-4f2f-a523-8f08ac37d025 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.409616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e3a88-81ca-4f2f-a523-8f08ac37d025 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.409616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e3a88-81ca-4f2f-a523-8f08ac37d025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.451273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ea8e3a88-81ca-4f2f-a523-8f08ac37d025 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:24.452648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ea8e3a88-81ca-4f2f-a523-8f08ac37d025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:24.462126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1120/2274] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:46:24.474876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:24.476567 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:24.554722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767859b5-6912-4016-b48e-da6bfe699897 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:24.558219 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39feed1e4a48413d823bf0ece9b8f59c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:24.558844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.559129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.559402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.569299 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:24.570743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f215d-b96b-4739-b99c-e2ed576d124e service glance] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:24.581840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1155/2275] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:46:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:46:24.746440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996155-768c-4c3c-9a98-61941935d6a0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.746760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996155-768c-4c3c-9a98-61941935d6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.747033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996155-768c-4c3c-9a98-61941935d6a0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.748404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1121/2276] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:46:24.753008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7af75-53b9-47e3-aae1-6b28a1d51fd6 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.753301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7af75-53b9-47e3-aae1-6b28a1d51fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.753578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7af75-53b9-47e3-aae1-6b28a1d51fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.791934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b5a7af75-53b9-47e3-aae1-6b28a1d51fd6 None None] MFA Rules not processed for user `0f2a170d431b4524b18178757fa3d627`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:24.793144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b5a7af75-53b9-47e3-aae1-6b28a1d51fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:24.799594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1156/2277] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:46:24.811842 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9996155-768c-4c3c-9a98-61941935d6a0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:24.822555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9996155-768c-4c3c-9a98-61941935d6a0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:24.826595 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:24.827181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.827457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.827724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.834614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:24.836036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8637065-a4ff-4efa-9377-132fe61ed1eb service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:24.843767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1122/2278] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:46:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:46:24.923132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5995efe-cf01-43ab-95c9-18b58b080d7a None None] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.923616 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5995efe-cf01-43ab-95c9-18b58b080d7a None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.923903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5995efe-cf01-43ab-95c9-18b58b080d7a None None] PATH_INFO: `/` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.925241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1157/2279] 10.222.0.39 () {54 vars in 921 bytes} [Wed Oct 19 13:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 13:46:24.930754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e8e1-aa9c-41fb-98f7-a88646e65828 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.931064 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e8e1-aa9c-41fb-98f7-a88646e65828 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.931364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e8e1-aa9c-41fb-98f7-a88646e65828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.951755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-51f9e8e1-aa9c-41fb-98f7-a88646e65828 None None] MFA Rules not processed for user `f025b86069754817868da5d0b19e3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:24.952946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-51f9e8e1-aa9c-41fb-98f7-a88646e65828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:24.962084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1123/2280] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Oct 19 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:46:24.972455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5995efe-cf01-43ab-95c9-18b58b080d7a None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:24.983986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5995efe-cf01-43ab-95c9-18b58b080d7a None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:24.987533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:24.988118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:24.988395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:24.988666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:24.999111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:25.001498 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:25.013046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1158/2281] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:46:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:46:27.467808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c3f-1ceb-4053-b469-79f4af34941e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:27.468290 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c3f-1ceb-4053-b469-79f4af34941e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:27.468388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c3f-1ceb-4053-b469-79f4af34941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:27.487240 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f82e0c3f-1ceb-4053-b469-79f4af34941e None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:27.488453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f82e0c3f-1ceb-4053-b469-79f4af34941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:27.495494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1124/2282] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:46:27.507131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:27.518591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c798c-0bf2-488b-85b8-157cf0a161e2 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:27.523545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:27.524133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:27.524410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:27.524680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:27.532990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:27.534566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:27.542683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1159/2283] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:46:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:46:37.482348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=21 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:37.539165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=56 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:48.836022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c50a-bafb-4e1e-938a-ff102bc6e40a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:48.836522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c50a-bafb-4e1e-938a-ff102bc6e40a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:48.836628 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766c50a-bafb-4e1e-938a-ff102bc6e40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:48.857048 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e766c50a-bafb-4e1e-938a-ff102bc6e40a None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:46:48.858230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e766c50a-bafb-4e1e-938a-ff102bc6e40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:46:48.865425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1125/2284] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:46:48.879273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:46:48.888401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a9a22-03ca-436c-9030-e029a86437e1 service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:46:48.893523 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:46:48.894118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:46:48.894397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:46:48.894706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:46:48.902662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:46:48.904719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:46:48.912898 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1160/2285] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:46:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:46:58.851733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:46:58.909493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:47:11.193355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:47:11.193972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:47:11.193972 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:47:11.213731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] MFA Rules not processed for user `e7a6ae7333d2422ebc9f616d482e527a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:47:11.214965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:47:11.270963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1126/2286] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:47:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:47:11.578152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:47:11.587359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d409baa-aebf-4120-b06f-931a3cb2bdc3 service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:47:11.590834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:47:11.591421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:47:11.591699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:47:11.591970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:47:11.593656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:47:11.602959 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:47:11.607416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:47:11.612996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1161/2287] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:47:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:47:21.264306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=25 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:47:21.609352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:47:33.685270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:47:33.695008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8e14-08a3-41f1-96c5-2540cd5dd213 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:47:33.698521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:47:33.699133 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:47:33.699418 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:47:33.700294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:47:33.702031 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:47:33.767186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:47:33.768642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:47:33.779437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1127/2288] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:47:35.679671 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:47:35.693205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b56f-2cfb-4a31-97e0-380459f74fc4 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:47:35.698689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:47:35.699298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:47:35.700139 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:47:35.700419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:47:35.702152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:47:35.714279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:47:35.716550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:47:35.727220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1162/2289] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:47:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:47:43.774721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:47:45.722724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:18.467403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:18.476005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70f90-0131-4b62-8092-607015b22be4 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:18.479649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:18.480267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:18.480549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:18.480822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:18.482529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:18.545265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:18.546710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:18.556216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1128/2290] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:21.857081 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:21.863122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:21.873644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecbf4e-1cda-473a-9ba8-564fdb1f0053 service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:21.878262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:21.878952 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:21.879230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:21.879611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] PATH_INFO: `/v3/users/6c828726ae814c0f879a5aa69f647a96` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:21.880970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c828726ae814c0f879a5aa69f647a96)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:21.883079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:22.083224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c828726ae814c0f879a5aa69f647a96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.090043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c828726ae814c0f879a5aa69f647a96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.102034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c828726ae814c0f879a5aa69f647a96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.102730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c828726ae814c0f879a5aa69f647a96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.166809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-0ad86033-3508-444a-aa87-b48ae314b8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c828726ae814c0f879a5aa69f647a96'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.172893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1163/2291] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:21 2022] DELETE /identity/v3/users/6c828726ae814c0f879a5aa69f647a96 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:22.409125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:22.421517 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58adae6-0c73-4387-8986-b83488180cfd service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:22.483692 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:22.484336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:22.484618 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:22.484894 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] PATH_INFO: `/v3/projects/bc8fa61ce6ff491b9d497db42d900f50` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:22.486128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8fa61ce6ff491b9d497db42d900f50)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:22.488104 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:22.608092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8fa61ce6ff491b9d497db42d900f50', 'request_id': 'req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.613850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8fa61ce6ff491b9d497db42d900f50', 'request_id': 'req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.686287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8fa61ce6ff491b9d497db42d900f50', 'request_id': 'req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.698317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] The token cache is being invalidate because project bc8fa61ce6ff491b9d497db42d900f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:22.698637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.699268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2dd902d5-6bb6-402b-bf2b-50ad78d36b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:22.700764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1129/2292] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:22 2022] DELETE /identity/v3/projects/bc8fa61ce6ff491b9d497db42d900f50 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:22.705609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:22.708695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:22.708695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:22.726237 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:22.727469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:22.785754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1164/2293] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:22.793837 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:22.794118 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:22.794390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:22.813389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:22.814608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:22.822282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1130/2294] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:22.826874 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:22.836488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97f5f5-c034-451d-b0d8-b166c124eeab None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:22.841642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:22.842236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:22.842536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:22.842834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:22.843482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:48:22.843792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:22.845408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:22.857390 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1165/2295] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:48:22.861501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:22.871241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992ae68-2e6e-4482-95e8-1ea3c1bcb141 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:22.875975 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:22.876565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:22.876844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:22.877244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:22.877866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:22.879861 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:22.895495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:23.076927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1131/2296] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:22 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:23.081278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.091770 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7c9d8-edcd-4cd6-8f57-860dca1972c6 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.147713 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.148351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.148633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.148905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.149568 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.152093 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.160033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:23.239370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1166/2297] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:23 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:23.243453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.254197 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e238e-0ce9-42b6-aa3b-10ac13faacc5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.258873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.263509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.263509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.263509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.263509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.263509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.272579 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1132/2298] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:23.277042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.287384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6076-2198-406a-aa36-3c6102d5a365 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.293082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.297920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.297920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.297920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.297920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.297920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7437cd6-e235-463e-8b1e-e435c8e94a0a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.306832 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1167/2299] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:23.310752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.320823 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3754e-ffc2-47b8-881c-285a4ca40fad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.325867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.326559 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.326847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.327174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] PATH_INFO: `/v3/projects/fc54cedfe83346edbcbace8d9a111de7/users/cc220444f0c4453cb8116a637375342f/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.342024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc54cedfe83346edbcbace8d9a111de7, user_id=cc220444f0c4453cb8116a637375342f, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.344773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.406914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1133/2300] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:23 2022] PUT /identity/v3/projects/fc54cedfe83346edbcbace8d9a111de7/users/cc220444f0c4453cb8116a637375342f/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:23.418131 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.418680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.419165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.446968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] MFA Rules not processed for user `cc220444f0c4453cb8116a637375342f`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:23.448670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:23.511202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1168/2301] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:23.515653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.576929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c80e1-ac00-4f9e-b111-a79b1c54dac6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.581843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.582387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.582707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.582987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.583630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.585548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.673797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1134/2302] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:23 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:48:23.677962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.688983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129977b-e679-4eac-a3c7-ae81a4c095f0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.744784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.745414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.745702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.745975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.746635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.748544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.824200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1169/2303] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:23 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:23.828364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.839114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2966-eaf7-43ab-b4cc-581cdede2ff2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.843641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.844300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.848276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.848276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.848276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.848276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:23.857981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1135/2304] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:23.862037 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:23.872814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4000de-4f58-4c9a-8a5b-0d857d55b938 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:23.877575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:23.878220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:23.878514 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:23.878808 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] PATH_INFO: `/v3/projects/4536636249674d37a4b09525adf4d8f6/users/869c4704fcc941e79a9704729fe494ad/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:23.894351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4536636249674d37a4b09525adf4d8f6, user_id=869c4704fcc941e79a9704729fe494ad, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:23.897117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.005255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1170/2305] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:23 2022] PUT /identity/v3/projects/4536636249674d37a4b09525adf4d8f6/users/869c4704fcc941e79a9704729fe494ad/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:24.009234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:24.020087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9720bc-8308-49eb-bc21-9063dedd897a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:24.076387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:24.076984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.077265 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.077540 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.078170 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:24.079786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431980-1dc9-4d27-ba22-e3366ac1ad7f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.089394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1136/2306] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:24.093490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:24.103723 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d53cd-ec9a-46e5-8784-0b46203c735a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:24.108342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:24.108987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.109267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.109540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] PATH_INFO: `/v3/projects/4536636249674d37a4b09525adf4d8f6/users/869c4704fcc941e79a9704729fe494ad/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.111463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4536636249674d37a4b09525adf4d8f6, user_id=869c4704fcc941e79a9704729fe494ad, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:24.114062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.124635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1171/2307] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:24 2022] PUT /identity/v3/projects/4536636249674d37a4b09525adf4d8f6/users/869c4704fcc941e79a9704729fe494ad/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:24.136502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.141659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.141659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.164650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] MFA Rules not processed for user `869c4704fcc941e79a9704729fe494ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:24.165850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:24.185518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:24.234479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1137/2308] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:24.246420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:24.248225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:24.271169 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:24.307802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c6b62-7287-4b1a-8946-7f1dcf7a0eb8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:24.312793 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:24.313404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.313689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.313965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.324440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:24.329333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d219da-751e-4800-ad86-ed208065020f service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.336838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1172/2309] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:24.403464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:24.465508 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cac4b2-3dba-45c0-85e7-8de895dcab77 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:24.470042 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:24.470707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.470989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.471271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] PATH_INFO: `/v3/users/cc220444f0c4453cb8116a637375342f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.472408 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc220444f0c4453cb8116a637375342f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:24.474321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.683122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc220444f0c4453cb8116a637375342f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:24.683749 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc220444f0c4453cb8116a637375342f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:24.690870 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc220444f0c4453cb8116a637375342f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:24.703980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc220444f0c4453cb8116a637375342f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:24.775792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc220444f0c4453cb8116a637375342f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:24.781750 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1138/2310] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:24 2022] DELETE /identity/v3/users/cc220444f0c4453cb8116a637375342f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:24.786244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.786563 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.786847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.807364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:24.808697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:24.865929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1173/2311] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:24.876203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:24.877632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:24.920043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:24.937194 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f810b-70a5-4046-abaf-5bab763c879e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:24.940674 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:24.941270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:24.941548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:24.941820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:24.950611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:24.952073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:24.962242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1139/2312] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:25.351153 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:25.352597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:25.371280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7ebb7-87b5-4c5a-913e-e3fa143e8c59 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:25.371280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:25.371280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:25.371280 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:25.371646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] PATH_INFO: `/v3/projects/fc54cedfe83346edbcbace8d9a111de7` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:25.372220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc54cedfe83346edbcbace8d9a111de7)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:25.374261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:25.492753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc54cedfe83346edbcbace8d9a111de7', 'request_id': 'req-3df6613f-3316-4302-8c32-e5b45447199a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.498434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc54cedfe83346edbcbace8d9a111de7', 'request_id': 'req-3df6613f-3316-4302-8c32-e5b45447199a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.571997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc54cedfe83346edbcbace8d9a111de7', 'request_id': 'req-3df6613f-3316-4302-8c32-e5b45447199a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.583353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] The token cache is being invalidate because project fc54cedfe83346edbcbace8d9a111de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:25.587154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.587154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.587154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1174/2313] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:25 2022] DELETE /identity/v3/projects/fc54cedfe83346edbcbace8d9a111de7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:25.590350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:25.600124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f1669-8ffb-4e7f-a362-b1565f809b83 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:25.656382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:25.657001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:25.657276 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:25.657547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] PATH_INFO: `/v3/users/869c4704fcc941e79a9704729fe494ad` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:25.658712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869c4704fcc941e79a9704729fe494ad)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:25.660625 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:25.837616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869c4704fcc941e79a9704729fe494ad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.838909 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869c4704fcc941e79a9704729fe494ad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.845900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869c4704fcc941e79a9704729fe494ad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.857905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869c4704fcc941e79a9704729fe494ad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.915382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0e5f0956-58f5-443f-b53c-9c0a8bc6ace0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869c4704fcc941e79a9704729fe494ad'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:25.921569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1140/2314] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:25 2022] DELETE /identity/v3/users/869c4704fcc941e79a9704729fe494ad => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:26.316995 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:26.318586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:26.332440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6613f-3316-4302-8c32-e5b45447199a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:26.377300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:26.390324 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:26.391006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.391297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.391564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] PATH_INFO: `/v3/projects/4536636249674d37a4b09525adf4d8f6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.392674 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4536636249674d37a4b09525adf4d8f6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:26.394610 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:26.509866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4536636249674d37a4b09525adf4d8f6', 'request_id': 'req-d530053b-9f35-41fe-9c8f-7545f5fa6e31'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:26.515637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4536636249674d37a4b09525adf4d8f6', 'request_id': 'req-d530053b-9f35-41fe-9c8f-7545f5fa6e31'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:26.583650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4536636249674d37a4b09525adf4d8f6', 'request_id': 'req-d530053b-9f35-41fe-9c8f-7545f5fa6e31'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:26.595909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] The token cache is being invalidate because project 4536636249674d37a4b09525adf4d8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:26.596132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:26.596755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d530053b-9f35-41fe-9c8f-7545f5fa6e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:26.598230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1175/2315] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:26 2022] DELETE /identity/v3/projects/4536636249674d37a4b09525adf4d8f6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:48:26.603169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.603950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.604234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.624533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:26.625756 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:26.675073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:26.684830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1141/2316] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:26.692557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.692847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.693130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.712655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:26.714679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:26.721735 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1176/2317] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:26.725849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:26.736561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514c4a-0e71-49f5-afe0-177d0ef8c894 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:26.741158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:26.743377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.743377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.743377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.743377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:48:26.743377 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:26.745492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:26.757140 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1142/2318] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:48:26.761394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:26.770656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c2668-ebfa-498a-ab3d-12cee0ee529f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:26.775352 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:26.775949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.776229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.776502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.777125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:26.779046 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:26.876091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1177/2319] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:26 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:26.880241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:26.890372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22477fab-a132-4b16-8443-fd98587273fd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:26.947340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:26.947980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:26.948262 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:26.948535 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:26.949168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:26.951117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.052916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1143/2320] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:26 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:27.056851 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.067293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0156885-6897-495c-891e-1f76d27e99ee admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.071830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.072421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.072701 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.073013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.073677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.075291 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.085014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1178/2321] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:27.089490 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.099588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad337230-c451-4625-ade7-c58a41e8d76e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.105077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.105664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.105942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.106213 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.106874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.108471 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea9ac-c71c-4c2a-a619-b3206b4dccc4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.118794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1144/2322] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:27.122787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.132576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ec1d-4d67-49b8-a57b-f4e0f7d02951 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.137088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.137721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.138000 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.138275 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] PATH_INFO: `/v3/projects/49d0711f1921494499eb73acc8d661c5/users/d5a5013b0f1b4be7a69af6d5e64b47a3/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.153197 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d0711f1921494499eb73acc8d661c5, user_id=d5a5013b0f1b4be7a69af6d5e64b47a3, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.155968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.240449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1179/2323] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:27 2022] PUT /identity/v3/projects/49d0711f1921494499eb73acc8d661c5/users/d5a5013b0f1b4be7a69af6d5e64b47a3/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:48:27.250868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.251158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.251438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.279902 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] MFA Rules not processed for user `d5a5013b0f1b4be7a69af6d5e64b47a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:27.281109 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:27.344199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1145/2324] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:27.348761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.409798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e8ed2-397b-4c03-8494-380ecedc5288 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.414695 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.415293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.415576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.415872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.416502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.418394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.503794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:27.511240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1180/2325] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:27 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:48:27.515462 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.526741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff651948-8bbe-48e2-b363-b16e0c30b005 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.574717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:27.583420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.584024 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.584305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.584581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.585216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.587169 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.681237 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1146/2326] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:27 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:27.685840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.696483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a3bb6-7eef-406b-bae9-607d061fa67f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.701779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.702406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.702721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.703003 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.703649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.705243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.715937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1181/2327] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:27.719783 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.729644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5594f42-6023-4457-b0c1-e136d40c69af admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.734779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.735407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.735739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.736014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.736667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.739054 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eff7d2-5d98-4584-9f6f-f1d29633a62e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.749547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1147/2328] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:27.753453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:27.764157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30560-5129-4266-a6b6-7314caa71a4e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:27.768866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:27.769506 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.769787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.770059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] PATH_INFO: `/v3/projects/f78a32a5dc1647569d850013f6009403/users/fccde784e77d4c959e7b94cf55de6632/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.784737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78a32a5dc1647569d850013f6009403, user_id=fccde784e77d4c959e7b94cf55de6632, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:27.787826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:27.890691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1182/2329] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:27 2022] PUT /identity/v3/projects/f78a32a5dc1647569d850013f6009403/users/fccde784e77d4c959e7b94cf55de6632/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:27.902259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:27.902807 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:27.902924 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:27.932266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] MFA Rules not processed for user `fccde784e77d4c959e7b94cf55de6632`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:27.933592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:27.997955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1148/2330] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:28.002677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.065882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800186c4-6300-4579-b16b-002221bb5338 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.070741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.071358 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.071642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.071925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.072546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.074436 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.219117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1183/2331] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:28 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:28.228515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.240066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b1533-1682-4a25-8d1b-e8dbda92bd56 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.297047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.297647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.297929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.298203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.298875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.300802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.372168 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1149/2332] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:28 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:28.376334 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.386909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132c4de-fd04-4f7f-b937-a65fc8f7c099 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.391650 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.392255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.392541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.392820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.393466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.395133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.407198 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1184/2333] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:28.411404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.422727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a3214-05ca-4bbb-b7b3-b50d7966c5c4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.430831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.430831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.430831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.430831 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] PATH_INFO: `/v3/projects/d482e62c39c144b3aba9bfb10bd6f696/users/bfd705a9485e4cc29bc152dd19c5febd/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.443550 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d482e62c39c144b3aba9bfb10bd6f696, user_id=bfd705a9485e4cc29bc152dd19c5febd, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.446204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.532528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1150/2334] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:28 2022] PUT /identity/v3/projects/d482e62c39c144b3aba9bfb10bd6f696/users/bfd705a9485e4cc29bc152dd19c5febd/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:48:28.536433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.547401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bd4bd-2a39-42c8-b260-01627f8bcfd9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.602200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.602823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.603103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.603373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.604001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.606209 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409eb7-a77a-4550-8696-b32ef8cee858 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.615760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1185/2335] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:28.619856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.629989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc8b4e-2f21-4f87-a301-0cfb0718494c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.634642 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.635279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.635555 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.635833 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] PATH_INFO: `/v3/projects/d482e62c39c144b3aba9bfb10bd6f696/users/bfd705a9485e4cc29bc152dd19c5febd/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.638267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d482e62c39c144b3aba9bfb10bd6f696, user_id=bfd705a9485e4cc29bc152dd19c5febd, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.640896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.650828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1151/2336] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:28 2022] PUT /identity/v3/projects/d482e62c39c144b3aba9bfb10bd6f696/users/bfd705a9485e4cc29bc152dd19c5febd/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:28.661368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.661657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.662181 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.689144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] MFA Rules not processed for user `bfd705a9485e4cc29bc152dd19c5febd`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:28.690338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:28.710107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:28.757260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1186/2337] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:28.769601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.771303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:28.786641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:28.828939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd9df4-4692-4042-9105-074428f9cc15 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.832586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.833178 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.833572 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.833848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:28.843592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:28.845090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:28.854664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1152/2338] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:28.944927 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:28.946308 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:28.958580 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fda5f-fc28-4a17-a16f-273a3b2370aa None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:28.962123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:28.962779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:28.963058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:28.963329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:29.019245 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:29.020725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4f49c-0519-4f13-b323-9ee06c53e99a service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:29.030278 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1187/2339] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:33.912225 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:33.975085 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa470373-58b2-4f5e-b204-505896b38f4e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:33.979801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:33.980461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:33.980738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:33.981009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] PATH_INFO: `/v3/users/d5a5013b0f1b4be7a69af6d5e64b47a3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:33.982130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a5013b0f1b4be7a69af6d5e64b47a3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:33.984148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:34.153682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a5013b0f1b4be7a69af6d5e64b47a3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.154350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a5013b0f1b4be7a69af6d5e64b47a3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.161659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a5013b0f1b4be7a69af6d5e64b47a3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.174714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a5013b0f1b4be7a69af6d5e64b47a3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.240687 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a5013b0f1b4be7a69af6d5e64b47a3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.247832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1153/2340] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:33 2022] DELETE /identity/v3/users/d5a5013b0f1b4be7a69af6d5e64b47a3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:34.252332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:34.253264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:34.253264 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:34.272331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:34.273522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:34.334293 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1188/2341] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:34.347412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:34.347412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:34.387933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:34.410858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30419ae-9a48-4045-a8c4-308bfde75a42 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:34.410858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:34.410858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:34.410858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:34.411589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:34.420521 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:34.421935 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:34.431883 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1154/2342] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:34.836771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:34.838368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:34.850376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c4713-d824-45e8-a37a-71cbb05c38d0 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:34.855042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:34.856161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:34.856439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:34.856715 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] PATH_INFO: `/v3/projects/49d0711f1921494499eb73acc8d661c5` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:34.857816 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d0711f1921494499eb73acc8d661c5)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:34.859958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:34.995338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d0711f1921494499eb73acc8d661c5', 'request_id': 'req-940b41bc-b9aa-40a5-a7c1-eaa426208b66'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:34.997719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:35.001633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d0711f1921494499eb73acc8d661c5', 'request_id': 'req-940b41bc-b9aa-40a5-a7c1-eaa426208b66'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.068978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d0711f1921494499eb73acc8d661c5', 'request_id': 'req-940b41bc-b9aa-40a5-a7c1-eaa426208b66'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.081909 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] The token cache is being invalidate because project 49d0711f1921494499eb73acc8d661c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:35.082254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.082942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.084434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1189/2343] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:34 2022] DELETE /identity/v3/projects/49d0711f1921494499eb73acc8d661c5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:35.088452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:35.098115 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92568ac6-62ca-4581-8884-110ccea090b3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:35.157092 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:35.157715 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:35.157992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:35.158281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] PATH_INFO: `/v3/users/fccde784e77d4c959e7b94cf55de6632` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:35.159554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccde784e77d4c959e7b94cf55de6632)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:35.161502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:35.334020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccde784e77d4c959e7b94cf55de6632'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.334699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccde784e77d4c959e7b94cf55de6632'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.342219 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccde784e77d4c959e7b94cf55de6632'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.354253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccde784e77d4c959e7b94cf55de6632'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.427283 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccde784e77d4c959e7b94cf55de6632'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:35.434033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1155/2344] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:35 2022] DELETE /identity/v3/users/fccde784e77d4c959e7b94cf55de6632 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:35.859642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:35.861241 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:35.875154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b41bc-b9aa-40a5-a7c1-eaa426208b66 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:35.933446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:35.934121 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:35.934405 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:35.934751 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] PATH_INFO: `/v3/projects/f78a32a5dc1647569d850013f6009403` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:35.935854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78a32a5dc1647569d850013f6009403)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:35.938089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:36.049419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78a32a5dc1647569d850013f6009403', 'request_id': 'req-7d58aac0-af63-45f9-8f83-ff54678e37ce'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.055040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78a32a5dc1647569d850013f6009403', 'request_id': 'req-7d58aac0-af63-45f9-8f83-ff54678e37ce'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.109211 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78a32a5dc1647569d850013f6009403', 'request_id': 'req-7d58aac0-af63-45f9-8f83-ff54678e37ce'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.121143 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] The token cache is being invalidate because project f78a32a5dc1647569d850013f6009403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:36.121396 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.122020 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.123624 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1190/2345] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:35 2022] DELETE /identity/v3/projects/f78a32a5dc1647569d850013f6009403 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:36.127529 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:36.129080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:36.142454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecb2b0-54ab-4cb5-938d-b8e4e1c52dbe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:36.194752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:36.200994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:36.201621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:36.201901 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:36.202296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] PATH_INFO: `/v3/users/bfd705a9485e4cc29bc152dd19c5febd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:36.203565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd705a9485e4cc29bc152dd19c5febd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:36.205533 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:36.385776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd705a9485e4cc29bc152dd19c5febd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.386393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd705a9485e4cc29bc152dd19c5febd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.394223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd705a9485e4cc29bc152dd19c5febd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.406731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd705a9485e4cc29bc152dd19c5febd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.463427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc863116-5648-4956-9262-e18711ceca48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd705a9485e4cc29bc152dd19c5febd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:36.470049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1156/2346] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:36 2022] DELETE /identity/v3/users/bfd705a9485e4cc29bc152dd19c5febd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:36.882611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:36.884044 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:36.898688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58aac0-af63-45f9-8f83-ff54678e37ce admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:36.957464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:36.958134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:36.958415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:36.958725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] PATH_INFO: `/v3/projects/d482e62c39c144b3aba9bfb10bd6f696` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:36.959846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d482e62c39c144b3aba9bfb10bd6f696)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:36.961839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:36.973337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:37.067835 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd482e62c39c144b3aba9bfb10bd6f696', 'request_id': 'req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:37.073284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd482e62c39c144b3aba9bfb10bd6f696', 'request_id': 'req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:37.135890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd482e62c39c144b3aba9bfb10bd6f696', 'request_id': 'req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:37.147646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] The token cache is being invalidate because project d482e62c39c144b3aba9bfb10bd6f696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:37.148011 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:37.148571 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5a45d830-05b7-4f03-971e-f4ed9d1cd1c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:37.150122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1191/2347] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:36 2022] DELETE /identity/v3/projects/d482e62c39c144b3aba9bfb10bd6f696 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:37.156236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.156731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.156848 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.177431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:37.178549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:37.239603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1157/2348] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:37.248600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.248886 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.249159 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.269383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:37.270589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:37.279635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1192/2349] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:37.283927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.294412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de927d-6f1e-41b0-b4ad-ff1f10050e35 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.299038 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.299632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.299914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.300188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.300830 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:48:37.301129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.302746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.314090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1158/2350] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 13:48:37.319519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.328811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44713026-c664-45e6-8377-cf6db4bcd8a3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.333783 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.334406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.334760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.335077 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.335724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.337690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.513944 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1193/2351] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:37 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:37.515027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.525087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966dcbe-5632-4479-af97-dd1e6fffe77d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.580701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.581299 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.581578 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.581850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.582477 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.584415 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.594791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:37.720634 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1159/2352] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:37 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:37.724668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.735311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147087bc-f21e-4e3f-83cd-b0c725c8cfd7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.739752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.740346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.740625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.740896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.741554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.743325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.753620 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1194/2353] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:37.761632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.772311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab033d5-6533-4161-b302-1be65471224e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.776940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.778445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.778813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.779344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.780186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.782049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb95ea6-c250-4fe4-87bd-25701e2124d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.792968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1160/2354] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:37.796887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:37.806811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb16c9b-c050-4a61-952c-59a3186816d6 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:37.811349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:37.811986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.812267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.812545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] PATH_INFO: `/v3/projects/577c93362eb74f56b9a708130f1e6384/users/26f10b8d1d6b457b97c1f15aa6b81d1a/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.827258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=577c93362eb74f56b9a708130f1e6384, user_id=26f10b8d1d6b457b97c1f15aa6b81d1a, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:37.829872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:37.907987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1195/2355] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:37 2022] PUT /identity/v3/projects/577c93362eb74f56b9a708130f1e6384/users/26f10b8d1d6b457b97c1f15aa6b81d1a/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:37.920446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:37.920739 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:37.921017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:37.948433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] MFA Rules not processed for user `26f10b8d1d6b457b97c1f15aa6b81d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:37.950129 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:38.017273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1161/2356] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:38.022284 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.083611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097998c3-eb37-472d-afa1-6418e4467f7e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.088221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.088812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.089091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.089363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.090010 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.091945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.195845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1196/2357] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:38 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:38.200657 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.211306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a13449-ac03-4334-9b5b-8d022212982a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.269574 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.270191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.270475 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.270778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.271428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.273372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.374086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1162/2358] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:38 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:38.384609 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.395428 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb3859-9eeb-4ac8-9072-f4a86d676eef admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.400122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.400741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.401025 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.401300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.401962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.403570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.414401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1197/2359] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:38.418998 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.428917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4a8f2-ffcd-46ce-952f-3dd912b0851e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.433506 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.434730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.435016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.435292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] PATH_INFO: `/v3/projects/792328669aea417580208c2baeeeae34/users/b84430f580c54877bad8be9338592a6b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.449814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792328669aea417580208c2baeeeae34, user_id=b84430f580c54877bad8be9338592a6b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.452489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.514032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1163/2360] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:38 2022] PUT /identity/v3/projects/792328669aea417580208c2baeeeae34/users/b84430f580c54877bad8be9338592a6b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:38.517947 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.527879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787adbf-f430-4781-985d-7e021409a1b0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.535661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:38.583521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.584117 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.584398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.584678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.585311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.586932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89515762-f885-43f1-a74c-2f1941150c2c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.596445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1198/2361] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:38.600709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.611738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1236f46-2165-4124-9020-469e272c4351 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.616403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.617007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.617286 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.617562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] PATH_INFO: `/v3/projects/792328669aea417580208c2baeeeae34/users/b84430f580c54877bad8be9338592a6b/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.619485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=792328669aea417580208c2baeeeae34, user_id=b84430f580c54877bad8be9338592a6b, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.622752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.632889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1164/2362] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:38 2022] PUT /identity/v3/projects/792328669aea417580208c2baeeeae34/users/b84430f580c54877bad8be9338592a6b/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:38.643089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522891b-f2d8-422b-b819-5545523507af None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.643409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522891b-f2d8-422b-b819-5545523507af None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.643688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522891b-f2d8-422b-b819-5545523507af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.671965 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6522891b-f2d8-422b-b819-5545523507af None None] MFA Rules not processed for user `b84430f580c54877bad8be9338592a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:38.673167 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6522891b-f2d8-422b-b819-5545523507af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:38.740202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1199/2363] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:38.751723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:38.753413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:38.781829 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:38.812659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a3d96-6462-4fc5-aad8-3ff2fc3eef2a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:38.816495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:38.820321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:38.820321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:38.820321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:38.826791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:38.828220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f6978-5156-44e8-a50e-c77927e27baa service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:38.837889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1165/2364] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:39.326996 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522891b-f2d8-422b-b819-5545523507af None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:39.389270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522891b-f2d8-422b-b819-5545523507af None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:39.394111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:39.394810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:39.395094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:39.395371 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] PATH_INFO: `/v3/users/26f10b8d1d6b457b97c1f15aa6b81d1a` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:39.396520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f10b8d1d6b457b97c1f15aa6b81d1a)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:39.398460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:39.542760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:39.551647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f10b8d1d6b457b97c1f15aa6b81d1a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:39.558301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f10b8d1d6b457b97c1f15aa6b81d1a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:39.570455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f10b8d1d6b457b97c1f15aa6b81d1a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:39.571151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f10b8d1d6b457b97c1f15aa6b81d1a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:39.640072 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f10b8d1d6b457b97c1f15aa6b81d1a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:39.645984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1200/2365] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:39 2022] DELETE /identity/v3/users/26f10b8d1d6b457b97c1f15aa6b81d1a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:39.650532 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:39.651032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:39.651142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:39.670560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:39.671785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:39.729785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1166/2366] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:39.739290 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:39.740727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:39.797683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064a119-c320-4321-8fc2-379389710dd7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:39.801325 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:39.801917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:39.802200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:39.802463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:39.811746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:39.813157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:39.822344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1201/2367] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:40.205186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:40.207931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:40.219520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d83558-3b23-48f6-952a-de95166bac7b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:40.224012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:40.224649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:40.224929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:40.225206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] PATH_INFO: `/v3/projects/577c93362eb74f56b9a708130f1e6384` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:40.226333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=577c93362eb74f56b9a708130f1e6384)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:40.228277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:40.329509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '577c93362eb74f56b9a708130f1e6384', 'request_id': 'req-cb1dd655-16f9-4e7a-ae76-211604b9c71e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.335148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '577c93362eb74f56b9a708130f1e6384', 'request_id': 'req-cb1dd655-16f9-4e7a-ae76-211604b9c71e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.409463 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '577c93362eb74f56b9a708130f1e6384', 'request_id': 'req-cb1dd655-16f9-4e7a-ae76-211604b9c71e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.421797 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] The token cache is being invalidate because project 577c93362eb74f56b9a708130f1e6384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:40.422098 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.425553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.425553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1167/2368] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:40 2022] DELETE /identity/v3/projects/577c93362eb74f56b9a708130f1e6384 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:40.428401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:40.438047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04ddd-b689-4f7e-a85f-6dc305427656 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:40.493577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:40.494202 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:40.494487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:40.494797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] PATH_INFO: `/v3/users/b84430f580c54877bad8be9338592a6b` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:40.495915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84430f580c54877bad8be9338592a6b)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:40.497852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:40.716658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84430f580c54877bad8be9338592a6b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.723704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84430f580c54877bad8be9338592a6b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.724836 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:40.736036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84430f580c54877bad8be9338592a6b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.739809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84430f580c54877bad8be9338592a6b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.815204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f55b4fd7-9847-4fbb-8bab-e4b68d2a581a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84430f580c54877bad8be9338592a6b'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:40.821306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1202/2369] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:40 2022] DELETE /identity/v3/users/b84430f580c54877bad8be9338592a6b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:41.228497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:41.230035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:41.243556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dd655-16f9-4e7a-ae76-211604b9c71e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:41.293791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:41.303504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:41.304181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.304460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.304731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] PATH_INFO: `/v3/projects/792328669aea417580208c2baeeeae34` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.305919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792328669aea417580208c2baeeeae34)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:41.307846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:41.432671 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792328669aea417580208c2baeeeae34', 'request_id': 'req-e7bce649-6e78-4765-bc71-db1f175fb0f7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:41.438974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792328669aea417580208c2baeeeae34', 'request_id': 'req-e7bce649-6e78-4765-bc71-db1f175fb0f7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:41.501319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792328669aea417580208c2baeeeae34', 'request_id': 'req-e7bce649-6e78-4765-bc71-db1f175fb0f7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:41.512600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] The token cache is being invalidate because project 792328669aea417580208c2baeeeae34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:41.519340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:41.519340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7bce649-6e78-4765-bc71-db1f175fb0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:41.519340 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1168/2370] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:41 2022] DELETE /identity/v3/projects/792328669aea417580208c2baeeeae34 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:41.519934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.520393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.520518 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.539903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:41.541292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:41.599856 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1203/2371] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:41.610379 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.610701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.610982 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.630820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:41.632016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:41.639928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1169/2372] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:41.643872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:41.653458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf8ca5-3539-4f09-920c-8ef81f0d472c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:41.657968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:41.658594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.658876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.659149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.659792 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:48:41.660092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:41.661938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:41.673864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1204/2373] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:48:41.677755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:41.687640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4f0e0-def3-4ae5-aad4-41a3f787113e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:41.692409 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:41.693035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.693344 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.693619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.694254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:41.696258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:41.781128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1170/2374] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:41 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:41.782693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:41.793232 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba980910-15e2-467b-a2ab-2c7a76c8db86 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:41.849926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:41.850598 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.850883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.851161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.851796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:41.853745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:41.970966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1205/2375] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:41 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:41.974952 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:41.986394 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4f34-94ef-4168-a1d1-8856a7bf280f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:41.991011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:41.991742 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:41.992015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:41.992284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:41.992917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:41.994495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.004846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1171/2376] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:42.008923 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.019556 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252ae7-dd73-407c-af82-7a77b2a7c53c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.024287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.024928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.025215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.025491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.026134 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.027809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4616e54-481c-43fa-afe2-2c3b39ed5206 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.039458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1206/2377] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:42.043597 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.054229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b8f61-7787-479f-9a42-e5ef0ee5231a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.058808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.059453 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.059838 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.060035 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] PATH_INFO: `/v3/projects/2988d34ba5934d9086270902d510a1a1/users/7109443c239543a19207e76fa2bfe24f/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.074260 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2988d34ba5934d9086270902d510a1a1, user_id=7109443c239543a19207e76fa2bfe24f, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.077427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.154992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1172/2378] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:42 2022] PUT /identity/v3/projects/2988d34ba5934d9086270902d510a1a1/users/7109443c239543a19207e76fa2bfe24f/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:42.165493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.165785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.166063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.193133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] MFA Rules not processed for user `7109443c239543a19207e76fa2bfe24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:42.194332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:42.222964 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:42.257175 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1207/2379] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:42.261828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.323930 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d63d50-6056-455b-8694-532ae86c88fc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.329033 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.329630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.329907 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.330176 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.330850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.332731 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.424000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1173/2380] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:48:42 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:42.427993 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.439213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93fb60-f9eb-449d-91c4-cf51e2f1575e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.494058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.494693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.494987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.495251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.495868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.497779 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.588840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1208/2381] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:48:42 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:48:42.592700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.598805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:42.603874 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c0057-8073-4261-85b1-34654f102823 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.608461 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.609088 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.609369 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.609646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.610288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.611949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.622365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1174/2382] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:42.626846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.636670 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334955a9-3026-44df-a08e-626a0fedd0d7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.641904 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.642577 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.642862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.643140 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] PATH_INFO: `/v3/projects/d888062cbde0414cb8ffb17b802fa0dd/users/c0d183587ad0443dbb6431bc9f4c4464/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.657863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d888062cbde0414cb8ffb17b802fa0dd, user_id=c0d183587ad0443dbb6431bc9f4c4464, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.660530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.703915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1209/2383] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:42 2022] PUT /identity/v3/projects/d888062cbde0414cb8ffb17b802fa0dd/users/c0d183587ad0443dbb6431bc9f4c4464/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:42.707844 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.718284 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbf08b-6f4d-4207-ad40-2cc747eb39c4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.774814 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.775414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.775723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.775993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.776647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.778220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670c711-347a-492a-aed2-e9b93febce4c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.788495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1175/2384] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:48:42.792736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.803426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e7967-4766-4169-b092-1db527efa32b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:42.807932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:42.808573 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.808852 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.809129 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] PATH_INFO: `/v3/projects/d888062cbde0414cb8ffb17b802fa0dd/users/c0d183587ad0443dbb6431bc9f4c4464/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.811098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d888062cbde0414cb8ffb17b802fa0dd, user_id=c0d183587ad0443dbb6431bc9f4c4464, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:42.814337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:42.824297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1210/2385] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:48:42 2022] PUT /identity/v3/projects/d888062cbde0414cb8ffb17b802fa0dd/users/c0d183587ad0443dbb6431bc9f4c4464/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:42.834301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:42.836172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:42.836172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:42.862788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] MFA Rules not processed for user `c0d183587ad0443dbb6431bc9f4c4464`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:42.863915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:42.933772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1176/2386] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:48:42.945536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:42.947536 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:43.014761 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63fbb5-742c-4861-a166-e5647ee08bfe admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:43.018740 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:43.019471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:43.019753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:43.020027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:43.055204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:43.077834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:43.079331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:43.088574 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1211/2387] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:51.323527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:51.333130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6337b2c-7547-41d3-bd93-e9e9dc44d803 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:51.337089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:51.337700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:51.337976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:51.338242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:51.348052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:51.349469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76154b60-cbf2-4d00-8e7c-746a082dbeeb service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:51.359255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1177/2388] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:48:53.112364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=11 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:58.383772 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:58.448811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464c7c3-ffa6-4401-bf6b-b2289d2878a5 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:58.453591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:58.454228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:58.455210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:58.455489 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] PATH_INFO: `/v3/users/7109443c239543a19207e76fa2bfe24f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:58.456630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7109443c239543a19207e76fa2bfe24f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:58.458581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:58.642404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7109443c239543a19207e76fa2bfe24f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:58.650125 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7109443c239543a19207e76fa2bfe24f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:58.662402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7109443c239543a19207e76fa2bfe24f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:58.663123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7109443c239543a19207e76fa2bfe24f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:58.737709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7109443c239543a19207e76fa2bfe24f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:58.744170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1212/2389] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:58 2022] DELETE /identity/v3/users/7109443c239543a19207e76fa2bfe24f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:58.748679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:58.749175 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:58.751912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:58.769070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:48:58.770348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:58.822018 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:58.835492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1178/2390] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:48:58.846071 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:58.847850 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:58.913382 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d7db-33b8-464e-ae0f-8b0ea5f32755 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:58.917192 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:58.917815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:58.918273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:58.918586 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:58.930310 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:58.931784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:58.942110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1213/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:48:59.429931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:59.433811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:48:59.445278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb83f93-8a0c-4a1c-8b1d-77603f90396c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:59.449923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:59.451148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:59.451459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:59.451918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] PATH_INFO: `/v3/projects/2988d34ba5934d9086270902d510a1a1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:59.453157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2988d34ba5934d9086270902d510a1a1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:59.455793 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:59.543445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2988d34ba5934d9086270902d510a1a1', 'request_id': 'req-84e53c15-4266-4ec7-b502-faddab40ee1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.549094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2988d34ba5934d9086270902d510a1a1', 'request_id': 'req-84e53c15-4266-4ec7-b502-faddab40ee1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.599363 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2988d34ba5934d9086270902d510a1a1', 'request_id': 'req-84e53c15-4266-4ec7-b502-faddab40ee1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.610616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] The token cache is being invalidate because project 2988d34ba5934d9086270902d510a1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:48:59.610958 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.616459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.616459 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1179/2392] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:48:59 2022] DELETE /identity/v3/projects/2988d34ba5934d9086270902d510a1a1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:48:59.617279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:48:59.627496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebde85-ff03-4791-a299-0cbe381181cd service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:48:59.646267 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:48:59.683745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:48:59.684689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:48:59.684971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:48:59.685254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] PATH_INFO: `/v3/users/c0d183587ad0443dbb6431bc9f4c4464` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:48:59.686391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d183587ad0443dbb6431bc9f4c4464)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:48:59.688346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:48:59.860339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d183587ad0443dbb6431bc9f4c4464'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.868311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d183587ad0443dbb6431bc9f4c4464'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.881583 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d183587ad0443dbb6431bc9f4c4464'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.882304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d183587ad0443dbb6431bc9f4c4464'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.955190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e98af933-845d-40ef-b468-a27d96fec45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d183587ad0443dbb6431bc9f4c4464'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:48:59.961594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1214/2393] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:48:59 2022] DELETE /identity/v3/users/c0d183587ad0443dbb6431bc9f4c4464 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:00.452292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:00.453815 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:00.466948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e53c15-4266-4ec7-b502-faddab40ee1f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:00.524279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:00.524933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:00.525211 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:00.525479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] PATH_INFO: `/v3/projects/d888062cbde0414cb8ffb17b802fa0dd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:00.526609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d888062cbde0414cb8ffb17b802fa0dd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:00.528497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:00.639643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd888062cbde0414cb8ffb17b802fa0dd', 'request_id': 'req-157b4251-e580-4f70-9466-3e5c20e7f5a6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:00.645356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd888062cbde0414cb8ffb17b802fa0dd', 'request_id': 'req-157b4251-e580-4f70-9466-3e5c20e7f5a6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:00.690650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd888062cbde0414cb8ffb17b802fa0dd', 'request_id': 'req-157b4251-e580-4f70-9466-3e5c20e7f5a6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:00.701813 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] The token cache is being invalidate because project d888062cbde0414cb8ffb17b802fa0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:00.702105 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:00.702765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-157b4251-e580-4f70-9466-3e5c20e7f5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:00.704802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1180/2394] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:00 2022] DELETE /identity/v3/projects/d888062cbde0414cb8ffb17b802fa0dd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:00.710972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:00.711455 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:00.711582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:00.731234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:00.732345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:00.789587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1215/2395] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:49:00.852067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:00.852330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:00.852609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:00.872234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:00.873444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:00.881358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1181/2396] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:00.885634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:00.896014 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d5ec-ad14-4f2c-ba01-f69b27aa20e7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:00.900653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:00.901256 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:00.901541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:00.901817 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:00.902464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:49:00.902805 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:00.904393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:00.916984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1216/2397] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:49:00.921474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:00.930672 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e76b25-ae7a-42c9-bcc9-d30d829b848b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:00.935318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:00.935917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:00.936190 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:00.936457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:00.937077 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:00.938990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:00.943469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:01.034314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1182/2398] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:00 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:01.038891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.048939 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ac09-5bba-48aa-89b6-706c8522b4de admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.074350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:01.104434 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.105041 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.105327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.105605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.106243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.108205 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.220055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1217/2399] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:01 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:01.221231 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.232126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36facc6f-1db7-4292-a133-fd485eb2c71c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.237083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.237681 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.237962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.238249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.238927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.240503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.250805 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1183/2400] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:01.255339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.265977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a43b7-cb62-4cc3-87bc-23ba85728b23 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.271281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.271891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.272180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.272459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.273107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.274739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22656910-d6f0-49cc-937f-a28a568ed349 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.288844 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1218/2401] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:01.289900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.300396 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d7d9-5617-4059-98f7-b18727f7de13 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.304987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.305636 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.305916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.306188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] PATH_INFO: `/v3/projects/6c52d24b770d4e79a8e07e3bf9f89bcd/users/ddc4b4b4ee7d4052ae6c6b49fbc5e5b9/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.321594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c52d24b770d4e79a8e07e3bf9f89bcd, user_id=ddc4b4b4ee7d4052ae6c6b49fbc5e5b9, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.324298 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.411743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1184/2402] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:01 2022] PUT /identity/v3/projects/6c52d24b770d4e79a8e07e3bf9f89bcd/users/ddc4b4b4ee7d4052ae6c6b49fbc5e5b9/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:01.422863 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be913a58-6c54-4726-b211-f67af24e041c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.423173 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be913a58-6c54-4726-b211-f67af24e041c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.423505 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be913a58-6c54-4726-b211-f67af24e041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.451364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-be913a58-6c54-4726-b211-f67af24e041c None None] MFA Rules not processed for user `ddc4b4b4ee7d4052ae6c6b49fbc5e5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:01.452572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-be913a58-6c54-4726-b211-f67af24e041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:01.518984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1219/2403] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:01.530273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.531992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:01.591784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788eeb-42bf-423e-97d6-fb0e3642fbad admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.595651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.596250 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.596528 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.596840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.605656 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.607258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a54859-34e9-4ac2-9880-bc5f2d53ef5a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.616633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1185/2404] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:01.639309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be913a58-6c54-4726-b211-f67af24e041c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:01.702193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be913a58-6c54-4726-b211-f67af24e041c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:01.707236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:01.707872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:01.708160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:01.708437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] PATH_INFO: `/v3/users/ddc4b4b4ee7d4052ae6c6b49fbc5e5b9` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:01.709604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc4b4b4ee7d4052ae6c6b49fbc5e5b9)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:01.711575 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:01.922894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:01.929439 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc4b4b4ee7d4052ae6c6b49fbc5e5b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:01.936566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc4b4b4ee7d4052ae6c6b49fbc5e5b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:01.948519 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc4b4b4ee7d4052ae6c6b49fbc5e5b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:01.951552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc4b4b4ee7d4052ae6c6b49fbc5e5b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.045924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc4b4b4ee7d4052ae6c6b49fbc5e5b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.053130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1220/2405] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:01 2022] DELETE /identity/v3/users/ddc4b4b4ee7d4052ae6c6b49fbc5e5b9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:02.057693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:02.057980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:02.058251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:02.074124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:02.075522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:02.077757 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:02.079006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:02.139686 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1186/2406] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:02.149229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c24d-4799-42d3-8eec-eb7c92678d49 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:02.150614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:02.152251 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:02.154651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:02.155513 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:02.155755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:02.156031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:02.159755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:02.172383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:02.175282 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:02.187322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1221/2407] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:49:02.216011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435e8bd-248c-4484-b348-0d61096fbdfe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:02.220398 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:02.221005 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:02.221281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:02.221547 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:02.228425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:02.230868 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:02.232373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29454b45-d095-4837-9f0c-602be3eee396 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:02.241921 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1187/2408] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:02.738937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:02.740416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:02.755217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51539039-1957-40b8-b0e7-5b2b7a79f7e2 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:02.759786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:02.763165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:02.763165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:02.763165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] PATH_INFO: `/v3/projects/6c52d24b770d4e79a8e07e3bf9f89bcd` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:02.763165 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c52d24b770d4e79a8e07e3bf9f89bcd)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:02.764149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:02.862570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c52d24b770d4e79a8e07e3bf9f89bcd', 'request_id': 'req-4d421688-f523-4ff7-88ed-d8e94c7b1480'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.868055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c52d24b770d4e79a8e07e3bf9f89bcd', 'request_id': 'req-4d421688-f523-4ff7-88ed-d8e94c7b1480'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.942991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c52d24b770d4e79a8e07e3bf9f89bcd', 'request_id': 'req-4d421688-f523-4ff7-88ed-d8e94c7b1480'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.955272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] The token cache is being invalidate because project 6c52d24b770d4e79a8e07e3bf9f89bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:02.955559 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.956183 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4d421688-f523-4ff7-88ed-d8e94c7b1480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:02.957675 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1222/2409] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:02 2022] DELETE /identity/v3/projects/6c52d24b770d4e79a8e07e3bf9f89bcd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:02.962610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:02.963113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:02.963232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:02.982910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:02.984043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:03.046454 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1188/2410] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:03.054736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.055028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.055305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.075261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:03.076465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:03.084591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1223/2411] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:03.088889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.098811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b007b6-edaa-4e93-a4de-9fa3d3355f05 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.103357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.107419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.119081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1189/2412] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:49:03.124830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.134743 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3878720-eaf2-4429-b19f-e237616c17a8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.139414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.140013 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.140285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.140560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.141234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.143171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.253453 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:03.259666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1224/2413] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:03.263922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.273753 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4bbbb-18d8-467e-9a29-48ffc1aba2ed admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.333709 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.334331 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.334652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.334925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.335648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.337553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.434028 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1190/2414] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:03.438005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.448285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae23d5-32e5-4916-a103-33bba298129f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.452864 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.453457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.453734 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.454004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.454689 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.456327 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.466185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1225/2415] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:03.470433 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.480424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecf0e8-999a-47e2-8d9a-b4805909a998 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.485957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.486621 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.486906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.487173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.487811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.489381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e7671-cc2b-41d4-9dab-80dfcfd49c23 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.500698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1191/2416] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:03.505023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.515214 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1017c9e-77b3-49cf-a6be-bf45b8069305 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.519978 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.520623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.520906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.521185 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] PATH_INFO: `/v3/projects/96ca4ca3d8cd4faa961d1880d1116c8d/users/f642c0ddbb2c4f7093a9f4f5365360fb/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.536500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ca4ca3d8cd4faa961d1880d1116c8d, user_id=f642c0ddbb2c4f7093a9f4f5365360fb, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.539189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.646053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1226/2417] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:03 2022] PUT /identity/v3/projects/96ca4ca3d8cd4faa961d1880d1116c8d/users/f642c0ddbb2c4f7093a9f4f5365360fb/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:03.656130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.656427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.656733 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.687127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] MFA Rules not processed for user `f642c0ddbb2c4f7093a9f4f5365360fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:03.688399 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:03.741045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:03.758385 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1192/2418] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:03.762833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:03.830570 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bef435-571e-4195-a966-b8998f366ea2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:03.835226 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:03.835837 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:03.836184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:03.836461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:03.837101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:03.839780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:03.990200 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1227/2419] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:03.994497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.005268 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9341d5a-f522-4e83-8287-c8d6cd031d02 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.063685 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.067809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.067809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.067809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.067809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.067809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.167427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1193/2420] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:03 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:49:04.168949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.179919 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5325f5-2caa-4af6-a9c3-98c41e00999a admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.184907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.185516 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.185797 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.186073 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.186754 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.188984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.200654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1228/2421] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:04.205391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.215570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bfae6-c80d-42bc-b04a-e290ab757174 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.220404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.221072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.221354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.221631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] PATH_INFO: `/v3/projects/d65462755f8c4853a67ddce22de00010/users/849ef208c0f440258e38bdc3ebd94a6f/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.237474 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65462755f8c4853a67ddce22de00010, user_id=849ef208c0f440258e38bdc3ebd94a6f, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.241652 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.333554 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1194/2422] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:04 2022] PUT /identity/v3/projects/d65462755f8c4853a67ddce22de00010/users/849ef208c0f440258e38bdc3ebd94a6f/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:49:04.337680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.349158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fa877-499b-42c9-a489-ac44b06c0bea admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.410023 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.410798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.411332 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.411605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.412249 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.413843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e3d7c-2214-4270-92eb-14308e6268f4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.426540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1229/2423] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:04.428124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.439084 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06734431-e93a-4ecc-b4fd-6c7a6e96b0d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.443932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.444577 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.444852 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.445120 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] PATH_INFO: `/v3/projects/d65462755f8c4853a67ddce22de00010/users/849ef208c0f440258e38bdc3ebd94a6f/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.447123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65462755f8c4853a67ddce22de00010, user_id=849ef208c0f440258e38bdc3ebd94a6f, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.449719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.460626 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1195/2424] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:04 2022] PUT /identity/v3/projects/d65462755f8c4853a67ddce22de00010/users/849ef208c0f440258e38bdc3ebd94a6f/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:04.470881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.471174 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.471449 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.505018 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] MFA Rules not processed for user `849ef208c0f440258e38bdc3ebd94a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:04.506281 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:04.530789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:04.578942 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1230/2425] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:04.589735 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:04.591442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:04.651698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4baa16-0830-4dfa-ba43-a6f9b8d1fa3b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:04.655443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:04.656047 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:04.656330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:04.656606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:04.713996 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:04.715496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37562af0-0b84-4b9a-a64e-2b2ec2f26014 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:04.719049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:04.724832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1196/2426] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:08.829074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:08.892054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e5c2a-d11b-475a-acc4-8e827076dc8f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:08.896699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:08.897329 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:08.897613 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:08.897893 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] PATH_INFO: `/v3/users/f642c0ddbb2c4f7093a9f4f5365360fb` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:08.899054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f642c0ddbb2c4f7093a9f4f5365360fb)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:08.901007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:09.069928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f642c0ddbb2c4f7093a9f4f5365360fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.077101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f642c0ddbb2c4f7093a9f4f5365360fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.089691 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f642c0ddbb2c4f7093a9f4f5365360fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.090413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f642c0ddbb2c4f7093a9f4f5365360fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.165272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f642c0ddbb2c4f7093a9f4f5365360fb'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.171800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1231/2427] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:08 2022] DELETE /identity/v3/users/f642c0ddbb2c4f7093a9f4f5365360fb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:09.176224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:09.176774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:09.176774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:09.196021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:09.197241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:09.254875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1197/2428] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:09.264394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:09.265780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:09.308987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:09.323283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfe0fa-c542-44c1-b5a0-52a2a8b45fa8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:09.326215 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:09.326840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:09.327118 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:09.327386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:09.337151 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:09.338589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:09.348180 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1232/2429] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:09.715760 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:09.720184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:09.729630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d682c-3b49-47f5-b23d-a6bcfc73c31a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:09.734279 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:09.734938 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:09.735222 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:09.735497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] PATH_INFO: `/v3/projects/96ca4ca3d8cd4faa961d1880d1116c8d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:09.737112 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ca4ca3d8cd4faa961d1880d1116c8d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:09.739055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:09.856476 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ca4ca3d8cd4faa961d1880d1116c8d', 'request_id': 'req-99bf7afd-8456-4efb-bd60-8a367d5841ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.862630 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ca4ca3d8cd4faa961d1880d1116c8d', 'request_id': 'req-99bf7afd-8456-4efb-bd60-8a367d5841ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.919082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ca4ca3d8cd4faa961d1880d1116c8d', 'request_id': 'req-99bf7afd-8456-4efb-bd60-8a367d5841ce'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.931198 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] The token cache is being invalidate because project 96ca4ca3d8cd4faa961d1880d1116c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:09.931483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.932161 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:09.933637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1198/2430] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:09 2022] DELETE /identity/v3/projects/96ca4ca3d8cd4faa961d1880d1116c8d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:09.937676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:09.948223 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019be7-12a6-4c90-a682-e53ae3a33a79 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:10.004047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:10.004686 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:10.004972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:10.005250 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] PATH_INFO: `/v3/users/849ef208c0f440258e38bdc3ebd94a6f` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:10.006487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] RBAC: Authorizing `identity:delete_user(user_id=849ef208c0f440258e38bdc3ebd94a6f)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:10.008317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:10.217314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849ef208c0f440258e38bdc3ebd94a6f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.225048 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849ef208c0f440258e38bdc3ebd94a6f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.237908 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849ef208c0f440258e38bdc3ebd94a6f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.238736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849ef208c0f440258e38bdc3ebd94a6f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.329065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7cba5be5-038d-4f37-b888-e53f2105778c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849ef208c0f440258e38bdc3ebd94a6f'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.336221 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1233/2431] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:09 2022] DELETE /identity/v3/users/849ef208c0f440258e38bdc3ebd94a6f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:10.737896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:10.740655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:10.752407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7afd-8456-4efb-bd60-8a367d5841ce admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:10.811134 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:10.811772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:10.812058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:10.812336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] PATH_INFO: `/v3/projects/d65462755f8c4853a67ddce22de00010` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:10.813480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65462755f8c4853a67ddce22de00010)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:10.815939 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:10.840765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:10.943179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65462755f8c4853a67ddce22de00010', 'request_id': 'req-c62b95c7-a55f-4d3c-a739-8627911f11ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:10.948655 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65462755f8c4853a67ddce22de00010', 'request_id': 'req-c62b95c7-a55f-4d3c-a739-8627911f11ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:11.045362 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65462755f8c4853a67ddce22de00010', 'request_id': 'req-c62b95c7-a55f-4d3c-a739-8627911f11ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:11.057025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] The token cache is being invalidate because project d65462755f8c4853a67ddce22de00010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:11.057312 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:11.057962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c62b95c7-a55f-4d3c-a739-8627911f11ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:11.059481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1199/2432] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:10 2022] DELETE /identity/v3/projects/d65462755f8c4853a67ddce22de00010 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:11.064903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.065367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.065367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.085243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:11.086521 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:11.143879 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1234/2433] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:11.152721 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.153008 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.153281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.172796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:11.174016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:11.181929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1200/2434] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:11.185997 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.191354 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:11.197386 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c73149-3b6c-496e-9c6c-891f5f812516 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.205333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.205333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.205333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.205333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.205739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:49:11.205739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.206338 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.218471 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1235/2435] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:49:11.222062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.231209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2251c-a2bd-4b0a-8649-2dfab32c1535 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.239333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.239333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.239333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.239333 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.239751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.240086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.341390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1201/2436] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:11.345539 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.355668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dad1b9-42a0-4fe7-baee-25c39f11cce6 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.412627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.413254 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.413557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.413815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.414462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.416400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.529476 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1236/2437] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:11.536249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.546589 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0faf41-14bf-446c-9873-d2762125e306 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.551792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.552395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.552675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.552979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.553620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.555337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.566195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1202/2438] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:11.570099 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.580172 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf685b-ee7f-426d-a3ac-183858e6f466 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.585102 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.585867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.586148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.586422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.587098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.588721 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879df8-5115-4251-8f09-42b61d4d98d6 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.599976 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1237/2439] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:11.604177 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.614910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b49d9-2759-41eb-bdd4-4b793810d983 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.619393 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.620030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.620311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.620610 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] PATH_INFO: `/v3/projects/04687016d2384187833a489563e1c0e8/users/a1897f7c2afe4c0c957408f7191f2d3f/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.635273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04687016d2384187833a489563e1c0e8, user_id=a1897f7c2afe4c0c957408f7191f2d3f, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.637986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.717720 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1203/2440] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:11 2022] PUT /identity/v3/projects/04687016d2384187833a489563e1c0e8/users/a1897f7c2afe4c0c957408f7191f2d3f/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:11.728605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.728894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.729171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.756957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] MFA Rules not processed for user `a1897f7c2afe4c0c957408f7191f2d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:11.758166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:11.821248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1238/2441] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:11.825712 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:11.887886 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36fad6-5428-47c8-9559-3458f2f5b357 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:11.892425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:11.893025 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:11.893303 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:11.893576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:11.894209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:11.896157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:11.999679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1204/2442] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:11 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:49:12.003398 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.014416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603d171-b67e-4325-b34c-d2fc009a1cc8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.065458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:12.071410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.075421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.075421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.075421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.075421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.075421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.157298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1239/2443] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:12 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:12.162191 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.172520 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee8275-c877-46a5-b68a-23a3e201ce6d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.177255 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.177849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.178128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.178403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.179076 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.180664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.191223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1205/2444] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:12.195276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.205868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c75c29-edb6-481d-b1a1-29d26ac9ca9e admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.210435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.211063 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.211348 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.211627 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.212269 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.213855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4edd4-fbe8-4b52-87aa-06743001aec3 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.224376 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1240/2445] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:12.228313 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.238244 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a235c61-007f-47d1-8cb5-1c1c33620115 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.242864 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.243501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.243778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.244052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] PATH_INFO: `/v3/projects/bf1204aceab3497cbd22eca6448d5042/users/1ac849c409874652975a503915e7abd5/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.258243 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1204aceab3497cbd22eca6448d5042, user_id=1ac849c409874652975a503915e7abd5, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.260903 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.323684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:12.328980 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1206/2446] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:12 2022] PUT /identity/v3/projects/bf1204aceab3497cbd22eca6448d5042/users/1ac849c409874652975a503915e7abd5/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:12.339892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.340184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.340458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.367838 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] MFA Rules not processed for user `1ac849c409874652975a503915e7abd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:12.369035 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:12.434322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1241/2447] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:12.439381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.500702 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc9501-a565-4dcf-b633-029fe3a784d4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.505820 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.506425 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.506758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.507034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.507666 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.509560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.599594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1207/2448] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:12 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:12.603415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.614188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed5101-6242-4268-90fc-475abbbb9119 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.671787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.672400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.672683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.672958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.673601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.675538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.769088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1242/2449] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:12 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 13:49:12.773228 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.783776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f656b7-83cb-44d6-b5d8-dc308a316459 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.788955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.789679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.789968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.790246 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.790950 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.792564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.803858 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1208/2450] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:12.807913 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.818300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2d5b1-5caf-4c6a-944a-c2a67bf0c201 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:12.822876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:12.823517 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:12.823796 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:12.824065 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] PATH_INFO: `/v3/projects/fee209c58ea242a9ac93f8bd0b735465/users/4d8cd4a3f489409591c0d6a8e1082357/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:12.838406 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee209c58ea242a9ac93f8bd0b735465, user_id=4d8cd4a3f489409591c0d6a8e1082357, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:12.841049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:12.943088 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1243/2451] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:12 2022] PUT /identity/v3/projects/fee209c58ea242a9ac93f8bd0b735465/users/4d8cd4a3f489409591c0d6a8e1082357/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:49:12.946995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:12.956911 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b601921-3ac2-41f0-b611-269206e1f0d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:13.013738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:13.014342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:13.014649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:13.014965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:13.015596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:13.017166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be3cd4-d334-472c-b4d4-4046071dc4cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:13.027086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1209/2452] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:13.031062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:13.041525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548a0b7-0c16-43c8-aec8-80ea450f39f2 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:13.046017 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:13.046680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:13.046958 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:13.047230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] PATH_INFO: `/v3/projects/fee209c58ea242a9ac93f8bd0b735465/users/4d8cd4a3f489409591c0d6a8e1082357/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:13.049705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee209c58ea242a9ac93f8bd0b735465, user_id=4d8cd4a3f489409591c0d6a8e1082357, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:13.052385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:13.062696 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1244/2453] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:13 2022] PUT /identity/v3/projects/fee209c58ea242a9ac93f8bd0b735465/users/4d8cd4a3f489409591c0d6a8e1082357/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:13.074431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:13.074796 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:13.075078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:13.103207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] MFA Rules not processed for user `4d8cd4a3f489409591c0d6a8e1082357`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:13.104414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:13.163277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:13.172640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1210/2454] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:13.184602 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:13.186270 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:13.210402 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:13.244036 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee243218-676b-46da-b962-9ff4c5b049ed admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:13.248364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:13.248962 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:13.249243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:13.249515 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:13.305301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:13.306757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:13.315656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1245/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:16.338440 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:16.348834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc7963-c415-400c-93a2-b7bba834c673 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:16.352356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:16.352945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:16.353221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:16.353492 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:16.408632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:16.410063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:16.419897 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1211/2456] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:17.828866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:17.837720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfd387-a610-43d1-9b94-8309ea6ef479 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:17.841528 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:17.842138 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:17.842416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:17.842724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:17.853184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:17.854700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da8b42-edcc-46ea-be71-d0e8863a6b11 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:17.865058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1246/2457] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:23.464009 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:23.525920 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61b112-df25-4348-93ff-784551151415 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:23.530539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:23.534601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:23.534601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:23.534601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] PATH_INFO: `/v3/users/a1897f7c2afe4c0c957408f7191f2d3f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:23.534601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1897f7c2afe4c0c957408f7191f2d3f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:23.535592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:23.690675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1897f7c2afe4c0c957408f7191f2d3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:23.691241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1897f7c2afe4c0c957408f7191f2d3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:23.698119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1897f7c2afe4c0c957408f7191f2d3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:23.709984 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1897f7c2afe4c0c957408f7191f2d3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:23.775973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1897f7c2afe4c0c957408f7191f2d3f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:23.782137 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1212/2458] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:23 2022] DELETE /identity/v3/users/a1897f7c2afe4c0c957408f7191f2d3f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:23.787447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:23.787946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:23.788060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:23.807423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:23.808623 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:23.867645 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1247/2459] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:23.876889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:23.879390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:23.891933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:23.937510 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04eb31-8e7a-433c-8ba8-2a40b0904d94 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:23.941274 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:23.941821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:23.942101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:23.942375 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:23.951354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:23.952795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:23.962846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1213/2460] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:24.320787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:24.322431 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:24.337842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab2a0b-d8e3-4ae0-93fd-476481b039e5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:24.342716 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:24.343478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:24.343765 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:24.344042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] PATH_INFO: `/v3/projects/04687016d2384187833a489563e1c0e8` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:24.345259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04687016d2384187833a489563e1c0e8)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:24.354052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:24.468532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04687016d2384187833a489563e1c0e8', 'request_id': 'req-a310de36-f378-4553-8bcf-db7549ffa901'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.472421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:24.474727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04687016d2384187833a489563e1c0e8', 'request_id': 'req-a310de36-f378-4553-8bcf-db7549ffa901'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.564488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04687016d2384187833a489563e1c0e8', 'request_id': 'req-a310de36-f378-4553-8bcf-db7549ffa901'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.576651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] The token cache is being invalidate because project 04687016d2384187833a489563e1c0e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:24.577090 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.577877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.579469 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1248/2461] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:24 2022] DELETE /identity/v3/projects/04687016d2384187833a489563e1c0e8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:24.584263 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:24.594306 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7be66-0891-4ddf-b1e0-5b2cc5659985 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:24.650288 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:24.650949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:24.651226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:24.651497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] PATH_INFO: `/v3/users/1ac849c409874652975a503915e7abd5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:24.652592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac849c409874652975a503915e7abd5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:24.654562 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:24.814157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac849c409874652975a503915e7abd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.814157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac849c409874652975a503915e7abd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.821214 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac849c409874652975a503915e7abd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.833016 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac849c409874652975a503915e7abd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.890531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac849c409874652975a503915e7abd5'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:24.896766 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1214/2462] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:24 2022] DELETE /identity/v3/users/1ac849c409874652975a503915e7abd5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:25.286604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:25.288094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:25.301460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310de36-f378-4553-8bcf-db7549ffa901 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:25.357228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:25.357786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:25.358067 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:25.358339 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] PATH_INFO: `/v3/projects/bf1204aceab3497cbd22eca6448d5042` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:25.359486 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1204aceab3497cbd22eca6448d5042)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:25.361383 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:25.500726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1204aceab3497cbd22eca6448d5042', 'request_id': 'req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.506193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1204aceab3497cbd22eca6448d5042', 'request_id': 'req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.575865 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1204aceab3497cbd22eca6448d5042', 'request_id': 'req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.587432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] The token cache is being invalidate because project bf1204aceab3497cbd22eca6448d5042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:25.587725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.588359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.589875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1249/2463] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:25 2022] DELETE /identity/v3/projects/bf1204aceab3497cbd22eca6448d5042 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:49:25.594001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:25.595483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:25.608660 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e87e4c-d308-4c16-9631-5eb5a254bfa0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:25.637675 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:25.664122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:25.664747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:25.665023 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:25.665294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] PATH_INFO: `/v3/users/4d8cd4a3f489409591c0d6a8e1082357` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:25.666470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8cd4a3f489409591c0d6a8e1082357)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:25.668849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:25.822575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8cd4a3f489409591c0d6a8e1082357'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.823252 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8cd4a3f489409591c0d6a8e1082357'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.830227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8cd4a3f489409591c0d6a8e1082357'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.843127 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8cd4a3f489409591c0d6a8e1082357'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.936366 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ac90c9bd-9d66-4567-8006-8ddba39f2ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8cd4a3f489409591c0d6a8e1082357'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:25.942977 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1215/2464] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:49:25 2022] DELETE /identity/v3/users/4d8cd4a3f489409591c0d6a8e1082357 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:26.348937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:26.350429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:26.363664 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b09cd-1ceb-4ba7-b946-1b3a5e027bb0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:26.421355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:26.421994 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:26.422273 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:26.422579 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] PATH_INFO: `/v3/projects/fee209c58ea242a9ac93f8bd0b735465` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:26.423687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee209c58ea242a9ac93f8bd0b735465)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:26.426111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:26.443422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:26.570902 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee209c58ea242a9ac93f8bd0b735465', 'request_id': 'req-d0401189-faf6-43b6-a00d-1ee075e9a2b4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:26.576460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee209c58ea242a9ac93f8bd0b735465', 'request_id': 'req-d0401189-faf6-43b6-a00d-1ee075e9a2b4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:26.671831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee209c58ea242a9ac93f8bd0b735465', 'request_id': 'req-d0401189-faf6-43b6-a00d-1ee075e9a2b4'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:26.684385 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] The token cache is being invalidate because project fee209c58ea242a9ac93f8bd0b735465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:49:26.684666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:26.685295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d0401189-faf6-43b6-a00d-1ee075e9a2b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:49:26.686900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1250/2465] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:49:26 2022] DELETE /identity/v3/projects/fee209c58ea242a9ac93f8bd0b735465 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:49:26.691745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:26.692318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:26.692318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:26.712062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:26.713220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:26.772502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1216/2466] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:26.780554 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:26.780778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:26.781040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:26.801693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:26.802949 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:26.811560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1251/2467] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:26.814914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:26.824576 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cde23a-2076-4191-847e-7d0e29b8669d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:26.829093 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:26.829682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:26.829959 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:26.830232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:26.830906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:49:26.831207 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:26.832772 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:26.844404 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1217/2468] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:49:26.849005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:26.857991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0fba7-86e4-4777-aaba-37b8b02592d3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:26.863123 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:26.863712 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:26.863987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:26.864252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:26.864877 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:26.866781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:26.969840 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1252/2469] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:49:26 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:26.974493 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:26.984651 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebe96-5e42-4f92-b176-0ccf4e0d2569 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:27.023003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:27.043277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:27.043885 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.044167 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.044443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.045080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:27.047015 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:27.174527 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1218/2470] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:49:26 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:49:27.178587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:27.188708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b814-79a4-4455-889b-d65a4728e9a0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:27.193217 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:27.193815 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.194097 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.194369 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.195040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:27.196649 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:27.207016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1253/2471] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:27.210765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:27.220654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8af26-0d50-4ab1-bf22-2c518fe24b67 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:27.225801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:27.226387 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.226725 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.226999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.227631 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:27.229204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b4881-417d-4097-8ee4-e54e71fda274 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:27.238904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1219/2472] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:49:27.242869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:27.252572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc6861-7ce5-4515-b92d-7d9ff9c7deab admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:27.257054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:27.257690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.257971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.258244 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] PATH_INFO: `/v3/projects/152b4e4be03746349f79a6bd119c75a3/users/06f39f80c50f46e799e8ae23e9029d67/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.272956 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152b4e4be03746349f79a6bd119c75a3, user_id=06f39f80c50f46e799e8ae23e9029d67, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:27.275628 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:27.375798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1254/2473] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:49:27 2022] PUT /identity/v3/projects/152b4e4be03746349f79a6bd119c75a3/users/06f39f80c50f46e799e8ae23e9029d67/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:49:27.387526 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6d3a-319d-43ad-9307-f3476b25d62e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.387809 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6d3a-319d-43ad-9307-f3476b25d62e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.388081 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6d3a-319d-43ad-9307-f3476b25d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.416148 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57ad6d3a-319d-43ad-9307-f3476b25d62e None None] MFA Rules not processed for user `06f39f80c50f46e799e8ae23e9029d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:27.417352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57ad6d3a-319d-43ad-9307-f3476b25d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:27.480832 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1220/2474] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:27.491394 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:27.493050 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:27.549283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54939c41-3416-45d7-a36c-600319236998 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:27.552885 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:27.553483 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:27.553764 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:27.554040 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:27.562359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:27.563821 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:27.573055 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1255/2475] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:30.335808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1569168-51ec-4384-aade-688962e84fef None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:30.336300 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1569168-51ec-4384-aade-688962e84fef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:30.336402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1569168-51ec-4384-aade-688962e84fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:30.355456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b1569168-51ec-4384-aade-688962e84fef None None] MFA Rules not processed for user `06f39f80c50f46e799e8ae23e9029d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:30.356654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b1569168-51ec-4384-aade-688962e84fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:30.363483 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1221/2476] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:37.569092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:38.293936 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:38.295423 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:38.351130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee793cc6-d38d-4d77-bf63-0d257cb4a94f service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:38.354720 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:38.355322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:38.355622 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:38.355934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:38.357769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:38.439207 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:38.440653 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:38.460798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1256/2477] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:40.350728 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:43.460241 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1569168-51ec-4384-aade-688962e84fef None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:43.469822 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1569168-51ec-4384-aade-688962e84fef None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:43.473355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:43.473944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:43.474221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:43.474491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:43.476230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:43.547677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:43.549623 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe28cb-9ca5-4a89-b897-0727f2db8193 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:43.560812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1222/2478] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:44.716190 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:44.717753 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:44.728853 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3fcc-a868-44b5-9925-ab9466f2320e service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:44.732408 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:44.733001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:44.733276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:44.733543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:44.735204 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:44.748342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:44.751116 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:44.761133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1257/2479] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:53.555792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=34 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:54.756806 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=67 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:49:57.306716 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3faf-6400-44b8-84ef-b578e171e758 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:57.307188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3faf-6400-44b8-84ef-b578e171e758 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:57.307301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3faf-6400-44b8-84ef-b578e171e758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:57.326803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-122d3faf-6400-44b8-84ef-b578e171e758 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:57.327986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-122d3faf-6400-44b8-84ef-b578e171e758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:57.383030 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1223/2480] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:49:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:57.533566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:57.535261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:57.602657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156a0b5-4211-44be-a105-b622d97abc1d service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:57.607657 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:57.608260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:57.608546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:57.608824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:57.616916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:57.618491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:57.627301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1258/2481] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:49:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:49:58.435669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:58.436549 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:58.437012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:58.456810 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:58.458017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:58.465342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1224/2482] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:49:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:58.478220 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:58.488825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862b1f-7496-4310-8a3e-3bd246b87bee service swift] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:58.493871 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:58.494473 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:58.494869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:58.495148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:58.505341 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:58.506972 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918b70-3c23-4cc3-bc9a-99884fdddf36 service swift] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:58.515708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1259/2483] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:49:59.035441 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:59.045090 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5725c6e-d753-488b-b35c-788cf184801c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:59.048745 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:59.049352 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:59.049632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:59.049933 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:59.051701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:59.122050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:59.123518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:59.133603 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1225/2484] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:49:59.193210 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67042-80a5-4cdf-995b-cc23bd8154e3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:59.193732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67042-80a5-4cdf-995b-cc23bd8154e3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:59.193983 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67042-80a5-4cdf-995b-cc23bd8154e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:59.213105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9cb67042-80a5-4cdf-995b-cc23bd8154e3 None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:59.214307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9cb67042-80a5-4cdf-995b-cc23bd8154e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:59.221759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1260/2485] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Oct 19 13:49:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:59.234124 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:59.246481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f5f4e-e5a0-454d-8e39-e5a989778686 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:59.252136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:59.252850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:59.253141 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:59.253417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:59.261875 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:59.263524 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:59.271892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1226/2486] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:49:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:49:59.840309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:59.840309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:59.840309 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:59.857557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] MFA Rules not processed for user `612a5c64ff3d4ec1826cf002bec59e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:49:59.858775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:49:59.866380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1261/2487] 10.222.0.39 () {58 vars in 967 bytes} [Wed Oct 19 13:49:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:49:59.878968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:49:59.891488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478609f-50cf-485a-80f4-8439382941cb service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:49:59.895699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7498ce3216d1411c961279f321418b7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:49:59.896308 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:49:59.896593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:49:59.896869 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:49:59.906407 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:49:59.908781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:49:59.920163 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1227/2488] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Oct 19 13:49:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 13:50:02.502706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:02.504179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:02.609345 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5434d-0b3d-428a-a96d-5c763a8ee07c None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:02.614368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:02.615007 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:02.615311 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:02.619534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:02.619534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:02.629824 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:02.631427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0817fe9-98fe-4f23-9bd2-acd2d3c0da4f service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:02.641331 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1262/2489] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:50:02.687293 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:02.700019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f447461-3e0d-4513-84c4-bd791ea51fc9 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:02.706087 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:02.706889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:02.707266 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:02.707767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] PATH_INFO: `/v3/users/06f39f80c50f46e799e8ae23e9029d67` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:02.709522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f39f80c50f46e799e8ae23e9029d67)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:02.711698 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:02.926334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f39f80c50f46e799e8ae23e9029d67'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:02.927206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f39f80c50f46e799e8ae23e9029d67'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:02.936227 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f39f80c50f46e799e8ae23e9029d67'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:02.943143 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:02.952600 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f39f80c50f46e799e8ae23e9029d67'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.022701 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f39f80c50f46e799e8ae23e9029d67'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.029232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1228/2490] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:50:02 2022] DELETE /identity/v3/users/06f39f80c50f46e799e8ae23e9029d67 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:03.033932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:03.034702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:03.035057 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:03.055682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:50:03.056892 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:03.090496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:03.092032 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:03.110897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=100 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:03.125454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1263/2491] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:50:03.135975 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:03.137500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:03.162063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83315-73cd-4395-a0d7-3eee295b072d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:03.166066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:03.166786 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:03.166987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:03.167269 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:03.169484 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:03.200907 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec6ce7-bcf6-4495-8096-b3db2190fc16 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:03.205894 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:03.206535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:03.206876 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:03.207154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:03.218487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:03.220982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:03.231433 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1264/2492] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:50:03.241534 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:03.243361 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:03.255350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1229/2493] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 13:50:03.501297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:03.503122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:03.520295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6cab3-6363-4b00-8ff3-c07ee7f0177a service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:03.527429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:03.528452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:03.529283 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:03.529564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] PATH_INFO: `/v3/projects/152b4e4be03746349f79a6bd119c75a3` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:03.532748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=152b4e4be03746349f79a6bd119c75a3)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:03.535872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:03.633644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152b4e4be03746349f79a6bd119c75a3', 'request_id': 'req-f5be9237-7928-4c2f-864b-25f77c68b6e8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.639477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152b4e4be03746349f79a6bd119c75a3', 'request_id': 'req-f5be9237-7928-4c2f-864b-25f77c68b6e8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.735228 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152b4e4be03746349f79a6bd119c75a3', 'request_id': 'req-f5be9237-7928-4c2f-864b-25f77c68b6e8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.749395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] The token cache is being invalidate because project 152b4e4be03746349f79a6bd119c75a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:50:03.749395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.749905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:03.756925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1265/2494] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:50:03 2022] DELETE /identity/v3/projects/152b4e4be03746349f79a6bd119c75a3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:06.273460 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:06.275014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:06.363373 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780704c-f3ca-4d67-87cd-1da4962de3da service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:06.397942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:06.422030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:06.423861 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:06.455899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:06.456635 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:06.456991 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:06.457334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:06.459467 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:06.475565 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:06.478537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:06.499800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1230/2495] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:50:06.581317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be9237-7928-4c2f-864b-25f77c68b6e8 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:06.587928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:06.588637 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:06.588925 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:06.593694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] PATH_INFO: `/v3/users/e7a6ae7333d2422ebc9f616d482e527a` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:06.593694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a6ae7333d2422ebc9f616d482e527a)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:06.593694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:06.621530 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:06.856823 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a6ae7333d2422ebc9f616d482e527a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:06.864857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a6ae7333d2422ebc9f616d482e527a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:06.880236 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a6ae7333d2422ebc9f616d482e527a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:06.880987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a6ae7333d2422ebc9f616d482e527a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:06.958111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a6ae7333d2422ebc9f616d482e527a'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:06.964638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1266/2496] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:50:06 2022] DELETE /identity/v3/users/e7a6ae7333d2422ebc9f616d482e527a => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:07.250956 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:07.262986 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30055daa-42aa-4bf3-80b8-0f1fcc27824e service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:07.327384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:07.328184 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:07.328470 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:07.328747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] PATH_INFO: `/v3/projects/7a1c12d9a4454de59ed63d3cedb71a58` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:07.330596 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1c12d9a4454de59ed63d3cedb71a58)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:07.332513 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:07.469155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1c12d9a4454de59ed63d3cedb71a58', 'request_id': 'req-06a296f0-ef40-43e7-8355-90f1282872e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.476007 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1c12d9a4454de59ed63d3cedb71a58', 'request_id': 'req-06a296f0-ef40-43e7-8355-90f1282872e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.592119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1c12d9a4454de59ed63d3cedb71a58', 'request_id': 'req-06a296f0-ef40-43e7-8355-90f1282872e0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.606185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] The token cache is being invalidate because project 7a1c12d9a4454de59ed63d3cedb71a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:50:07.606584 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.607427 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.609343 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1231/2497] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:50:07 2022] DELETE /identity/v3/projects/7a1c12d9a4454de59ed63d3cedb71a58 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:07.613188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:07.614676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:07.628049 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f903e-8389-45d8-8526-53097c9f9e32 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:07.684446 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:07.685080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:07.685360 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:07.685642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] PATH_INFO: `/v3/users/a19083c9d88349a39f15ca600b326ebc` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:07.698891 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19083c9d88349a39f15ca600b326ebc)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:07.700810 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:07.908790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19083c9d88349a39f15ca600b326ebc'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.916094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19083c9d88349a39f15ca600b326ebc'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.928030 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19083c9d88349a39f15ca600b326ebc'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:07.928727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19083c9d88349a39f15ca600b326ebc'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.087767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-4e4178bc-c2ce-475c-9e16-a5acaf68491c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19083c9d88349a39f15ca600b326ebc'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.094276 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1267/2498] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:50:07 2022] DELETE /identity/v3/users/a19083c9d88349a39f15ca600b326ebc => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:08.324962 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:08.326403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:08.340172 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a296f0-ef40-43e7-8355-90f1282872e0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:08.390329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:08.397436 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:08.398062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:08.398345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:08.398670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] PATH_INFO: `/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:08.409752 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b7e26e5e4d40e98f0d2b9bb483665f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:08.411673 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:08.519319 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b7e26e5e4d40e98f0d2b9bb483665f', 'request_id': 'req-d6fed13c-591d-4bb3-990f-a316540d9854'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.524840 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b7e26e5e4d40e98f0d2b9bb483665f', 'request_id': 'req-d6fed13c-591d-4bb3-990f-a316540d9854'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.596272 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b7e26e5e4d40e98f0d2b9bb483665f', 'request_id': 'req-d6fed13c-591d-4bb3-990f-a316540d9854'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.610424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] The token cache is being invalidate because project b4b7e26e5e4d40e98f0d2b9bb483665f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:50:08.610761 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.611486 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:50:08.613080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1232/2499] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:50:08 2022] DELETE /identity/v3/projects/b4b7e26e5e4d40e98f0d2b9bb483665f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:50:11.994148 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:11.995193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:11.995193 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:12.008868 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:12.014529 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] MFA Rules not processed for user `c996056bd0354124ae74e79abf2753a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:50:12.015747 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:12.071288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1268/2500] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:50:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:50:18.607543 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:22.129597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=22 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:50:58.730495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:58.731981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:58.789144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fed13c-591d-4bb3-990f-a316540d9854 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:58.792751 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:58.793350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:58.793633 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:58.793908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:58.795616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:58.853700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:58.855151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:58.864561 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1233/2501] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:50:59.200132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:50:59.210304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699beb20-4cc0-45df-af46-1b24a06a2515 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:50:59.213867 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:50:59.214466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:50:59.214773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:50:59.215043 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:50:59.217005 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:50:59.227384 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:50:59.228807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:50:59.236869 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1269/2502] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:50:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:51:00.374951 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:00.376275 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:00.460066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12411e77-98b8-4d34-9491-cf46ab5767b1 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:00.466480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:00.467173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:00.467457 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:00.467727 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:00.469525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:00.482402 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:00.484099 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:00.496164 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1234/2503] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:51:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:51:05.767896 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:05.776775 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1141a-2f2a-418e-9de9-c702ba5e850b service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:05.780380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:05.781122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:05.781404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:05.781676 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:05.783659 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:05.845458 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:05.846924 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:05.856426 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1270/2504] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:51:10.491987 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:51:13.116019 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:13.117695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:13.131905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f37c-bc8b-4170-97ee-573c489a3371 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:13.136380 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:13.137017 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:13.137296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:13.137570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] PATH_INFO: `/v3/users/8f759f41b4d44a53a0b60443b4df0a75` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:13.138737 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f759f41b4d44a53a0b60443b4df0a75)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:13.140645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:13.380067 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f759f41b4d44a53a0b60443b4df0a75'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.381234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f759f41b4d44a53a0b60443b4df0a75'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.388224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f759f41b4d44a53a0b60443b4df0a75'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.400061 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f759f41b4d44a53a0b60443b4df0a75'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.463877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f759f41b4d44a53a0b60443b4df0a75'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.469887 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1235/2505] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:51:13 2022] DELETE /identity/v3/users/8f759f41b4d44a53a0b60443b4df0a75 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:51:13.693625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:13.703887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1540e33-8a87-46fb-88bf-0be5224083b3 service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:13.761564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:13.762191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:13.762474 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:13.762780 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] PATH_INFO: `/v3/projects/76384f17c9384958983b188d3617be7e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:13.763899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76384f17c9384958983b188d3617be7e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:13.765932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:13.882141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76384f17c9384958983b188d3617be7e', 'request_id': 'req-7f90cf9d-53b3-403d-a3a5-c2df5b179293'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.887897 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76384f17c9384958983b188d3617be7e', 'request_id': 'req-7f90cf9d-53b3-403d-a3a5-c2df5b179293'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.955727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76384f17c9384958983b188d3617be7e', 'request_id': 'req-7f90cf9d-53b3-403d-a3a5-c2df5b179293'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.967482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] The token cache is being invalidate because project 76384f17c9384958983b188d3617be7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:51:13.967771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.968403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:13.969890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1271/2506] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:51:13 2022] DELETE /identity/v3/projects/76384f17c9384958983b188d3617be7e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:51:13.973932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:13.975918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:13.988967 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8bf5d-459b-42d5-b492-d993fece3f76 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:14.046925 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:14.047560 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:14.047843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:14.048117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] PATH_INFO: `/v3/users/214b9e38cc194a98ae7767679e852761` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:14.049249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=214b9e38cc194a98ae7767679e852761)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:14.051215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:14.232990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214b9e38cc194a98ae7767679e852761'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.233640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214b9e38cc194a98ae7767679e852761'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.240619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214b9e38cc194a98ae7767679e852761'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.252602 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214b9e38cc194a98ae7767679e852761'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.254230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:51:14.292835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214b9e38cc194a98ae7767679e852761'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.299057 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1236/2507] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:51:13 2022] DELETE /identity/v3/users/214b9e38cc194a98ae7767679e852761 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:51:14.541576 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:51:14.546135 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:51:14.556790 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90cf9d-53b3-403d-a3a5-c2df5b179293 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:51:14.566690 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:51:14.616651 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:51:14.617344 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:51:14.617636 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:51:14.617918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] PATH_INFO: `/v3/projects/87f91edec8244c70ab273793676adfbf` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:51:14.619089 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f91edec8244c70ab273793676adfbf)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:51:14.621047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:51:14.698845 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f91edec8244c70ab273793676adfbf', 'request_id': 'req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.704435 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f91edec8244c70ab273793676adfbf', 'request_id': 'req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.755420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f91edec8244c70ab273793676adfbf', 'request_id': 'req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.767145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] The token cache is being invalidate because project 87f91edec8244c70ab273793676adfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:51:14.767432 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.768119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:51:14.769545 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1272/2508] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:51:14 2022] DELETE /identity/v3/projects/87f91edec8244c70ab273793676adfbf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:51:24.407294 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:51:24.763971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:19.259397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:19.261431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:19.317355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716f2df-0333-4cfb-a09d-afd96e0b6b92 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:19.364670 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:19.365296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:19.365575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:19.365849 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:19.367583 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:19.429525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:19.430906 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:19.441000 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1237/2509] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:22.898397 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:22.899915 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:22.914739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ea99d-f3c1-4b23-9a5a-a7f8d5a4d603 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:22.920082 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:22.920704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:22.920985 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:22.921259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] PATH_INFO: `/v3/users/c996056bd0354124ae74e79abf2753a8` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:22.922356 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c996056bd0354124ae74e79abf2753a8)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:22.924472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:23.127292 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c996056bd0354124ae74e79abf2753a8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.134251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c996056bd0354124ae74e79abf2753a8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.146709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c996056bd0354124ae74e79abf2753a8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.153258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c996056bd0354124ae74e79abf2753a8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.245243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-59549cc6-a039-4452-abd9-7794b8e4bf05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c996056bd0354124ae74e79abf2753a8'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.251466 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1273/2510] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 13:52:22 2022] DELETE /identity/v3/users/c996056bd0354124ae74e79abf2753a8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:52:23.480452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:23.492100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215643f4-ff50-42b3-afa9-f521f9763e50 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:23.550107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:23.550764 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:23.551045 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:23.551315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] PATH_INFO: `/v3/projects/723dfd3a60ee412b8569bb0a6081dc7a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:23.552413 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=723dfd3a60ee412b8569bb0a6081dc7a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:23.554372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:23.562697 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:23.690677 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '723dfd3a60ee412b8569bb0a6081dc7a', 'request_id': 'req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.696238 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '723dfd3a60ee412b8569bb0a6081dc7a', 'request_id': 'req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.759119 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '723dfd3a60ee412b8569bb0a6081dc7a', 'request_id': 'req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.770971 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] The token cache is being invalidate because project 723dfd3a60ee412b8569bb0a6081dc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 13:52:23.771258 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.771893 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2f8dbd71-6bc5-4367-895a-0db4fc00b0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 13:52:23.773371 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1238/2511] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 13:52:23 2022] DELETE /identity/v3/projects/723dfd3a60ee412b8569bb0a6081dc7a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 13:52:23.778481 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:23.778804 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:23.779079 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:23.799106 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:52:23.803054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:23.859973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1274/2512] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:52:23.868497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:23.868785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:23.869060 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:23.889302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:52:23.890491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:23.898388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1239/2513] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:52:23.903258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:23.913136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4de6e-a181-45cf-bbdf-098c60e22b90 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:23.917694 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:23.918306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:23.918724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:23.919001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:23.919773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 13:52:23.920078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:23.921656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:23.933098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1275/2514] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 13:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 13:52:23.937381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:23.946601 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0ba20-2a8d-4cd9-890b-a6c04d6ac1bb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:23.951703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:23.952291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:23.952570 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:23.952843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:23.953502 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:23.955419 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.069397 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1240/2515] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 13:52:23 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:52:24.074022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:24.084252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72237622-8273-4b26-9786-229bba614ec7 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:24.139318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:24.139912 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.140189 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.140462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.141086 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:24.143028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.150786 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:24.204928 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1276/2516] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 13:52:24 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 13:52:24.209395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:24.220381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986e72c-bfbe-45db-bdc2-9ca055d0c695 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:24.225128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:24.225722 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.226002 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.226278 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.226954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:24.228531 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.238942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1241/2517] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:52:24.248251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:24.258826 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6272a-49fd-416f-9d2e-34ea84a638be admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:24.264001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:24.264600 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.264881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.265155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.265801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:24.267445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976f2bb-132f-44d3-88a0-fe63e276f6fc admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.278906 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1277/2518] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 13:52:24.283431 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:24.293784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a4cfc-fa36-482a-bea5-434e0a1ba646 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:24.298564 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:24.299166 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.299451 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.299754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] PATH_INFO: `/v3/projects/86a09ef08cc44d4b98c86f0e430006f4/users/c8e5d4efe11f4f198a9b028b69c23ee6/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.316138 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a09ef08cc44d4b98c86f0e430006f4, user_id=c8e5d4efe11f4f198a9b028b69c23ee6, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:24.318834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.364955 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1242/2519] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 13:52:24 2022] PUT /identity/v3/projects/86a09ef08cc44d4b98c86f0e430006f4/users/c8e5d4efe11f4f198a9b028b69c23ee6/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 13:52:24.370350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960365bb-4ba9-4513-b953-d89dd252963b None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.370663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960365bb-4ba9-4513-b953-d89dd252963b None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.370941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960365bb-4ba9-4513-b953-d89dd252963b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.390419 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-960365bb-4ba9-4513-b953-d89dd252963b None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:52:24.391658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-960365bb-4ba9-4513-b953-d89dd252963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:24.450367 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1278/2520] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:52:24.460113 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:24.462755 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:24.519417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1c7cd-cb38-4e8d-834f-ae2478eaa80c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:24.523223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:24.523824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:24.524107 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:24.524383 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:24.534267 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:24.535726 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:24.545569 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1243/2521] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:29.612318 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d8ef-3a94-4867-ae4a-c7ac15f1b9ad None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:29.612875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d8ef-3a94-4867-ae4a-c7ac15f1b9ad None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:29.612875 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d8ef-3a94-4867-ae4a-c7ac15f1b9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:29.646463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-a0d4d8ef-3a94-4867-ae4a-c7ac15f1b9ad None None] MFA Rules not processed for user `c8e5d4efe11f4f198a9b028b69c23ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:52:29.647731 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a0d4d8ef-3a94-4867-ae4a-c7ac15f1b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:29.719677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1279/2522] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:52:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:52:29.735880 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:29.737320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:29.785835 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:29.821014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163e7d-788d-4aee-953e-2ad7edb573c1 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:29.827302 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:29.827949 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:29.828270 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:29.828536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:29.837131 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:29.838828 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:29.848536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1244/2523] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:33.032465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:33.032991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:33.032991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:33.057391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] MFA Rules not processed for user `c8e5d4efe11f4f198a9b028b69c23ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:52:33.058802 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:33.069557 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1280/2524] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:52:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:52:33.376850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:33.378256 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:33.390632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ded0-c7fb-4fe9-bd48-323cd278eae6 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:33.394156 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:33.394781 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:33.395062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:33.395336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:33.396988 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:33.406382 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:33.407824 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:33.415645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1245/2525] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:33.802053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:33.803708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:33.872076 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:33.877684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8b69-28ea-4d66-a936-41c4b1e29c0e None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:33.881409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:33.882015 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:33.882295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:33.882604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:33.884336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:33.895719 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:33.900477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:33.907355 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1281/2526] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:43.412117 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=24 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:43.958581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=4 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:52:44.895588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:52:44.899898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:44.913957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbae47-9d4d-41be-a7f2-48f0e622e28a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:52:44.919866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:52:44.922827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:52:44.922827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:52:44.922827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:52:44.922827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:52:44.934771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:52:44.936347 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51a118-e1a3-4a95-983d-8838c3950bd0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:52:44.946900 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1246/2527] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:52:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:52:54.942424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:54:06.482137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:54:06.484122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:54:06.565594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844a627-8da0-40b8-adf0-99f6c3c8f1f4 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:54:06.570582 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:54:06.571184 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:54:06.571462 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:54:06.571733 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:54:06.573440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:54:06.584562 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:54:06.586111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:54:06.595834 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1282/2528] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:54:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:54:16.591625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=37 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:54:18.374479 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:54:18.375011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:54:18.375123 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:54:18.394668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] MFA Rules not processed for user `c8e5d4efe11f4f198a9b028b69c23ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:54:18.395919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:54:18.403122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1247/2529] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:54:18.702625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:54:18.711846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec91fe-f2f6-491d-b89c-8bb81f46f809 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:54:18.715410 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:54:18.716004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:54:18.716285 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:54:18.716587 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:54:18.718272 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:54:18.728120 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:54:18.729526 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:54:18.737968 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1283/2530] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:54:28.389669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:54:28.733849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:54:33.156339 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:54:33.187816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52e708-c924-4ac6-8aaa-b4acdf5ce97a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:54:33.194217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:54:33.194217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:54:33.194650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:54:33.194992 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:54:33.196769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:54:33.224259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:54:33.225851 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:54:33.237438 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1248/2531] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:54:43.232174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=18 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:56:00.123422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:56:00.126391 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:00.205298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b58e-6ecc-4a00-84d3-b0adeb5edcaa service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:56:00.209199 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:56:00.209820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:56:00.210098 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:56:00.210368 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:56:00.212136 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:00.225494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:56:00.226954 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:56:00.237596 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1284/2532] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:56:01.804821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:56:01.824230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a24c8-d66a-4f79-9e29-8c8ceeee8a39 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:56:01.827779 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:56:01.828368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:56:01.828646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:56:01.828915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:56:01.830682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:01.843799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:56:01.845204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea250a-a29c-41db-85dc-611d50dab49f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:56:01.855910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1249/2533] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:56:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:56:03.235145 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:56:03.264722 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdbfa9-1a32-4254-aae0-1cc21b343908 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:56:03.271787 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:56:03.272421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:56:03.272702 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:56:03.273191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:56:03.280491 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:03.294656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:56:03.296480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:56:03.309933 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1285/2534] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:56:11.851132 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:56:13.303543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=41 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:56:16.860746 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75641780-17f4-4682-affc-0636fadffc41 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:56:16.861332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75641780-17f4-4682-affc-0636fadffc41 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:56:16.861332 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75641780-17f4-4682-affc-0636fadffc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:56:16.880334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-75641780-17f4-4682-affc-0636fadffc41 None None] MFA Rules not processed for user `c8e5d4efe11f4f198a9b028b69c23ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:56:16.881525 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75641780-17f4-4682-affc-0636fadffc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:16.888904 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1250/2535] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:56:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 13:56:20.887849 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:56:20.897564 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96d3ee-999f-4a6c-aaae-17bcaf3f698d service nova] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:56:20.902130 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:56:20.902762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:56:20.903045 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:56:20.903319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:56:20.905056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:56:20.928054 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:56:20.929492 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b0aff-d5bd-43df-93bf-0909a0da1540 service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:56:20.939594 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1286/2536] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:56:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:56:26.875209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:56:30.934379 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=11 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:57:47.139186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75641780-17f4-4682-affc-0636fadffc41 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:57:47.152321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75641780-17f4-4682-affc-0636fadffc41 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:57:47.158350 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:57:47.159102 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:57:47.159390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:57:47.159668 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:57:47.167780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:57:47.169360 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:57:47.179068 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1251/2537] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:57:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:57:57.174792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:58:08.704585 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:58:08.707757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:58:08.707757 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:58:08.724502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] MFA Rules not processed for user `c8e5d4efe11f4f198a9b028b69c23ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 13:58:08.725665 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 13:58:08.732416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1287/2538] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 13:58:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 13:58:09.031487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:58:09.041062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efc670-1e28-416d-bd80-45c9c00f3194 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:58:09.044667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:58:09.045248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:58:09.045523 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:58:09.045791 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:58:09.053106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:58:09.054539 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:58:09.062682 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1252/2539] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:58:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:58:09.478918 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:58:09.492477 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a79fa-273e-4a0e-907c-217d6ffeb77d None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:58:09.498404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:58:09.498984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:58:09.499370 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:58:09.499654 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:58:09.508621 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:58:09.510238 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:58:09.520641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1288/2540] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:58:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:58:19.058799 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:58:19.515917 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=9 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:58:20.459503 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:58:20.473744 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b909e-5020-4a0b-96b2-0d19389ab97e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:58:20.479374 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:58:20.479979 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:58:20.480365 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:58:20.480644 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:58:20.490062 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:58:20.491695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:58:20.502428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1253/2541] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:58:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:58:30.497899 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 13:59:12.444692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 13:59:12.455404 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108a70e-687c-4d57-af5f-45422007d521 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 13:59:12.461031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 13:59:12.461625 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 13:59:12.461905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 13:59:12.462179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 13:59:12.470685 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 13:59:12.472252 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 13:59:12.482110 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1289/2542] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 13:59:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 13:59:22.477846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=6 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:03.283789 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:03.293349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b423e0-5b9d-4009-91c5-669b7d260f1e service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:03.297106 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:03.297713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:03.297999 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:03.298273 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:03.307215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:03.308654 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:03.336611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1254/2543] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:06.808105 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:06.820091 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3b1b-b7b7-4f1b-9630-85267ba66529 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:06.826984 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:06.827604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:06.828009 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:06.828288 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:06.830078 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:06.842171 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:06.844258 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:06.862400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1290/2544] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:06.948357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:06.958464 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189844e5-9e0b-45f2-9615-340769e595d5 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:06.963432 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:06.964063 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:06.964346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:06.964653 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] PATH_INFO: `/v3/users/c8e5d4efe11f4f198a9b028b69c23ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:06.965747 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e5d4efe11f4f198a9b028b69c23ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:06.967703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:07.109889 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e5d4efe11f4f198a9b028b69c23ee6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.110443 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e5d4efe11f4f198a9b028b69c23ee6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.117489 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e5d4efe11f4f198a9b028b69c23ee6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.129983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e5d4efe11f4f198a9b028b69c23ee6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.193139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5435c3cd-0ba5-4505-8db9-d1811a031011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e5d4efe11f4f198a9b028b69c23ee6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.199186 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1255/2545] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:06 2022] DELETE /identity/v3/users/c8e5d4efe11f4f198a9b028b69c23ee6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:07.478158 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:07.488953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b1cf2-3929-449b-a42b-85e62d028734 service placement] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:07.545546 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:07.546179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:07.546459 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:07.546768 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] PATH_INFO: `/v3/projects/86a09ef08cc44d4b98c86f0e430006f4` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:07.547870 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a09ef08cc44d4b98c86f0e430006f4)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:07.549766 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:07.665031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a09ef08cc44d4b98c86f0e430006f4', 'request_id': 'req-877297ed-3345-422b-aaa1-fe6723a130ab'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.671064 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a09ef08cc44d4b98c86f0e430006f4', 'request_id': 'req-877297ed-3345-422b-aaa1-fe6723a130ab'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.756935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a09ef08cc44d4b98c86f0e430006f4', 'request_id': 'req-877297ed-3345-422b-aaa1-fe6723a130ab'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.782033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] The token cache is being invalidate because project 86a09ef08cc44d4b98c86f0e430006f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:07.782319 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.782966 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-877297ed-3345-422b-aaa1-fe6723a130ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:07.784495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1291/2546] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:07 2022] DELETE /identity/v3/projects/86a09ef08cc44d4b98c86f0e430006f4 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:07.789509 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01954610-e921-452e-899b-3436bb4523b7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:07.789811 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01954610-e921-452e-899b-3436bb4523b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:07.790066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01954610-e921-452e-899b-3436bb4523b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:07.847434 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-01954610-e921-452e-899b-3436bb4523b7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:07.848648 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-01954610-e921-452e-899b-3436bb4523b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:07.906817 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1256/2547] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:07.914501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:07.915001 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:07.915156 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:07.934298 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:07.935561 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:07.943494 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1292/2548] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:07.947450 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01954610-e921-452e-899b-3436bb4523b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:07.957496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01954610-e921-452e-899b-3436bb4523b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:07.962142 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:07.962765 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:07.963049 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:07.963898 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:07.964556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 14:00:07.964866 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:07.966446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:07.975314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:07.978495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1257/2549] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 14:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 14:00:07.985550 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:07.995132 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8063baa-a5e9-4bff-ac29-cb1fe2742240 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:07.999847 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.000445 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.000732 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.001008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.001641 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.003597 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.083380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1293/2550] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:07 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:08.087511 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.097966 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9a9c-2368-40a7-924f-f4ab67d8344d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.153801 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.154424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.154738 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.155011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.155639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.157536 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.241981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1258/2551] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:08 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 14:00:08.246718 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.256827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f557b-2030-473c-8a81-b874648a1c0b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.262152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.262785 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.263066 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.263337 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.263973 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.265611 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.275781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1294/2552] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:08.279707 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.289640 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7d325-ed7f-4cc2-bc8b-0ca4e6979ae6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.294160 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.294784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.295126 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.295447 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.296080 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.297646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37582f83-9d67-4afd-985b-cf84b256959f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.309622 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1259/2553] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:08.313688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.324150 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af3d3d-e006-4faa-b93d-2faed40300eb admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.328854 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.329506 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.329813 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.330087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] PATH_INFO: `/v3/projects/7a091bde8060401195865593b082202e/users/0754531faf8c45d2b1b81a7ba7e211fe/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.344703 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a091bde8060401195865593b082202e, user_id=0754531faf8c45d2b1b81a7ba7e211fe, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.347304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.349422 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:08.407990 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1295/2554] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:08 2022] PUT /identity/v3/projects/7a091bde8060401195865593b082202e/users/0754531faf8c45d2b1b81a7ba7e211fe/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:08.418498 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.418821 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.419100 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.447322 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] MFA Rules not processed for user `0754531faf8c45d2b1b81a7ba7e211fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:08.448515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:08.511390 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1260/2555] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:08.522926 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.524604 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:08.596255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d915b-e4fb-44fa-84e3-3a00fecf30b0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.601809 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f025b86069754817868da5d0b19e3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.602413 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.602728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.603006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.611188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.612767 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f0c31-327a-41b6-a3c7-3b0332311a86 service nova] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.622111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1296/2556] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:08.828277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:08.889465 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58566-12bc-4b58-b1cb-f3cf37d504d1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:08.894125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:08.894792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:08.895075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:08.895348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] PATH_INFO: `/v3/users/0754531faf8c45d2b1b81a7ba7e211fe` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:08.896456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0754531faf8c45d2b1b81a7ba7e211fe)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:08.898337 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:08.909221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:09.107585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0754531faf8c45d2b1b81a7ba7e211fe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.108215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0754531faf8c45d2b1b81a7ba7e211fe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.115154 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0754531faf8c45d2b1b81a7ba7e211fe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.127078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0754531faf8c45d2b1b81a7ba7e211fe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.196812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0754531faf8c45d2b1b81a7ba7e211fe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.203896 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1261/2557] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:08 2022] DELETE /identity/v3/users/0754531faf8c45d2b1b81a7ba7e211fe => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:09.208440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:09.208953 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:09.209074 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:09.228279 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:09.229484 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:09.288795 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1297/2558] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:09.298736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:09.300183 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:09.358130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0bd8b-a84d-426e-bceb-b7e50fe03b52 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:09.361827 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:09.362437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:09.362748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:09.363022 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:09.372181 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:09.373629 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb400f-a248-4155-9e71-ca8a09e7402a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:09.383050 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1262/2559] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:09.795031 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:09.796478 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:09.810534 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908a08-c94f-4ab6-8725-20bc681873d7 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:09.816552 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:09.817359 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:09.817646 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:09.817922 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] PATH_INFO: `/v3/projects/7a091bde8060401195865593b082202e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:09.819059 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a091bde8060401195865593b082202e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:09.820977 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:09.912881 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a091bde8060401195865593b082202e', 'request_id': 'req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.919080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a091bde8060401195865593b082202e', 'request_id': 'req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.976300 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a091bde8060401195865593b082202e', 'request_id': 'req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.988349 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] The token cache is being invalidate because project 7a091bde8060401195865593b082202e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:09.988635 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.989259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-7953e6b6-a98e-4632-9a8d-ec8a88d0d4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:09.990890 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1298/2560] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:09 2022] DELETE /identity/v3/projects/7a091bde8060401195865593b082202e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:09.996439 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:09.996729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:09.997003 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.017027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:10.018315 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:10.077065 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1263/2561] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:10.086323 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.086647 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.086930 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.094569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:10.107380 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:10.120149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:10.120149 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1299/2562] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:10.123095 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.132882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5097ac-b25e-4fca-99dd-b3046d342535 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.137487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.138078 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.138358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.138667 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.139314 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 14:00:10.139614 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.141424 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.153073 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1264/2563] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 14:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 14:00:10.157305 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.167543 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95ed4c-2a0f-4694-ae17-2c39b88f03ef None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.172342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.172937 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.173216 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.173488 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.174111 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.176233 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.283971 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1300/2564] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:10 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:10.288173 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.298873 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20bad-bbae-49e5-8271-3811d585d8fe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.311356 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:10.356704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.357353 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.357995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.358320 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.358976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.360941 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.465612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1265/2565] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:10 2022] POST /identity/v3/users => generated 303 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:10.469707 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.480709 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac93db-98b8-4842-a1a8-70bfad6f6131 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.485437 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.486038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.486322 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.486630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.487336 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.488941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.498656 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1301/2566] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:10.502321 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.512217 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c06b9d-4f88-4e58-afc4-fc23195c83a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.517114 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.517700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.517983 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.518264 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.518940 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.520518 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6c327-e753-4981-82bc-a111bd0e3883 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.530704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1266/2567] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:10.534591 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.545034 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c2c7e-3bdd-4d03-801c-6e64125968bd admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.549705 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.550346 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.550666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.550946 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] PATH_INFO: `/v3/projects/9c9c69f338984f0c8b60ae5abf1ddf6e/users/75d9b8c3c4db40c59af393ef70d90766/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.565411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9c69f338984f0c8b60ae5abf1ddf6e, user_id=75d9b8c3c4db40c59af393ef70d90766, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.568053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.642827 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1302/2568] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:10 2022] PUT /identity/v3/projects/9c9c69f338984f0c8b60ae5abf1ddf6e/users/75d9b8c3c4db40c59af393ef70d90766/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:10.647680 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1405-3655-486b-b7ab-207cc0b90464 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.647961 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1405-3655-486b-b7ab-207cc0b90464 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.648233 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1405-3655-486b-b7ab-207cc0b90464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.667568 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c64c1405-3655-486b-b7ab-207cc0b90464 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:10.668754 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c64c1405-3655-486b-b7ab-207cc0b90464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:10.726931 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1267/2569] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:10.736387 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:10.738075 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:10.794905 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4183899-1081-48f8-9866-097566c2fc20 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:10.798538 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:10.799142 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:10.799424 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:10.799699 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:10.808601 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:10.810019 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:10.819295 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1303/2570] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:12.017834 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.018384 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.018803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.048468 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] MFA Rules not processed for user `75d9b8c3c4db40c59af393ef70d90766`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:12.049736 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:12.236963 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1268/2571] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:12.241460 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.251687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c12b94-77aa-4363-88a1-0526f36c92ab service neutron] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.256170 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.256762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.257056 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.257317 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.257941 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.259884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.358242 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1304/2572] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:12 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:12.362364 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.373729 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591e7fa-9a39-451a-ba74-7e93bce81b11 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.397929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:12.430052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.430693 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.430976 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.431249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.431892 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.434348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.559412 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1269/2573] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:12 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:12.563501 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.573730 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70ac81-d5d7-4e84-bcc7-6a6e27d4bd4c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.578531 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.579146 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.579447 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.579708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.580351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.581969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.593094 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1305/2574] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:12.596417 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.606556 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fae13-1c8a-4e00-9e9d-4785253fb437 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.611108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.611740 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.612020 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.612291 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] PATH_INFO: `/v3/projects/40d7f973e34442d19a755ab690e290ad/users/b5a2aa563d9b4b8f9519c54b07a66900/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.627588 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d7f973e34442d19a755ab690e290ad, user_id=b5a2aa563d9b4b8f9519c54b07a66900, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.630224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.713318 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1270/2575] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:12 2022] PUT /identity/v3/projects/40d7f973e34442d19a755ab690e290ad/users/b5a2aa563d9b4b8f9519c54b07a66900/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 14:00:12.717304 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.727303 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda5e-e914-4cef-b9e6-9c500ba41ae5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.784038 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.784642 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.784929 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.785206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.785843 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.787467 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04a0d7-e258-4867-a04c-82df1402a509 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.797475 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1306/2576] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:12.803421 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:12.812790 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433358a-e74f-43c1-86f7-2dcd3f280937 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:12.817669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:12.818330 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:12.818646 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:12.818928 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] PATH_INFO: `/v3/projects/40d7f973e34442d19a755ab690e290ad/users/b5a2aa563d9b4b8f9519c54b07a66900/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:12.820843 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d7f973e34442d19a755ab690e290ad, user_id=b5a2aa563d9b4b8f9519c54b07a66900, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:12.823488 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:12.833620 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1271/2577] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:12 2022] PUT /identity/v3/projects/40d7f973e34442d19a755ab690e290ad/users/b5a2aa563d9b4b8f9519c54b07a66900/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:14.042920 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:14.046363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:14.046363 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:14.051454 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:14.072673 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] MFA Rules not processed for user `b5a2aa563d9b4b8f9519c54b07a66900`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:14.073882 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:14.149551 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1307/2578] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:15.557860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:15.557860 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:15.626641 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861680f3-a29a-45ff-9279-cd25a0bfd610 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:15.631669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:15.632223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:15.632504 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:15.632776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:15.634481 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:15.646185 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:15.647784 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:15.658158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1272/2579] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:15.690788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:15.754644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa8d2-6efa-40db-a714-5b242bc45389 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:15.759033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:15.759658 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:15.759935 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:15.760206 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] PATH_INFO: `/v3/users/75d9b8c3c4db40c59af393ef70d90766` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:15.761307 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d9b8c3c4db40c59af393ef70d90766)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:15.763257 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:15.955457 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d9b8c3c4db40c59af393ef70d90766'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:15.962393 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d9b8c3c4db40c59af393ef70d90766'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:15.974803 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d9b8c3c4db40c59af393ef70d90766'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:15.975495 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d9b8c3c4db40c59af393ef70d90766'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.028407 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d9b8c3c4db40c59af393ef70d90766'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.034739 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1308/2580] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:15 2022] DELETE /identity/v3/users/75d9b8c3c4db40c59af393ef70d90766 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:16.277990 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:16.289586 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c19b6-962a-4149-bff3-a67040e1c59a service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:16.346711 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:16.347328 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:16.347612 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:16.347891 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] PATH_INFO: `/v3/projects/9c9c69f338984f0c8b60ae5abf1ddf6e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:16.348997 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9c69f338984f0c8b60ae5abf1ddf6e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:16.350973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:16.361206 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:16.457323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9c69f338984f0c8b60ae5abf1ddf6e', 'request_id': 'req-2074ed5e-729a-462a-a953-c8d8cd4e0295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.463072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9c69f338984f0c8b60ae5abf1ddf6e', 'request_id': 'req-2074ed5e-729a-462a-a953-c8d8cd4e0295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.519146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9c69f338984f0c8b60ae5abf1ddf6e', 'request_id': 'req-2074ed5e-729a-462a-a953-c8d8cd4e0295'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.530581 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] The token cache is being invalidate because project 9c9c69f338984f0c8b60ae5abf1ddf6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:16.530812 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.531444 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.532923 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1273/2581] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:16 2022] DELETE /identity/v3/projects/9c9c69f338984f0c8b60ae5abf1ddf6e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:16.536986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:16.538401 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:16.551316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c2592-5247-4546-94cc-51acebfc274b admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:16.555938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:16.607841 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:16.608470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:16.608752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:16.609028 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] PATH_INFO: `/v3/users/b5a2aa563d9b4b8f9519c54b07a66900` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:16.610155 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a2aa563d9b4b8f9519c54b07a66900)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:16.612095 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:16.813333 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a2aa563d9b4b8f9519c54b07a66900'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.820429 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a2aa563d9b4b8f9519c54b07a66900'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.832667 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a2aa563d9b4b8f9519c54b07a66900'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.833377 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a2aa563d9b4b8f9519c54b07a66900'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.910427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-fc430d98-b6a9-4957-bb20-7cc267155696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a2aa563d9b4b8f9519c54b07a66900'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:16.916480 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1309/2582] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:16 2022] DELETE /identity/v3/users/b5a2aa563d9b4b8f9519c54b07a66900 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:17.151847 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.153929 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:17.166953 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074ed5e-729a-462a-a953-c8d8cd4e0295 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.224158 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.224785 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.225066 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.225341 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] PATH_INFO: `/v3/projects/40d7f973e34442d19a755ab690e290ad` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.226442 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d7f973e34442d19a755ab690e290ad)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.228338 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.327856 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d7f973e34442d19a755ab690e290ad', 'request_id': 'req-e5895196-1914-42ea-bacd-a71f2579c81d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:17.333381 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d7f973e34442d19a755ab690e290ad', 'request_id': 'req-e5895196-1914-42ea-bacd-a71f2579c81d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:17.407867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d7f973e34442d19a755ab690e290ad', 'request_id': 'req-e5895196-1914-42ea-bacd-a71f2579c81d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:17.418994 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] The token cache is being invalidate because project 40d7f973e34442d19a755ab690e290ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:17.419281 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:17.419908 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e5895196-1914-42ea-bacd-a71f2579c81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:17.421357 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1274/2583] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:17 2022] DELETE /identity/v3/projects/40d7f973e34442d19a755ab690e290ad => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:17.426342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.426666 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.426945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.446297 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:17.447525 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:17.504708 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1310/2584] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:17.514103 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.514391 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.514708 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.534684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:17.535876 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:17.543917 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1275/2585] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 14:00:17.547916 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.557442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6284ba-d2a3-4b61-b703-c9c076380c12 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.561982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.562605 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.562887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.563161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.563801 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 14:00:17.564101 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.565677 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.577364 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1311/2586] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 14:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 14:00:17.581452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.591149 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b1e8b-d9c3-4f5e-8ec8-c4bb527e6d4c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.595762 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.596355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.596637 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.596912 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.597537 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.599487 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.683403 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:17.689355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1276/2587] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:17 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:17.693565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.704306 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f42d55-08f2-43f8-aa7c-23dceb7204d5 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.715884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:17.761440 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.762655 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.762934 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.763213 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.763846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.765741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.870403 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1312/2588] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:17 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 14:00:17.875021 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.885224 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2d3f6-95f7-4007-b053-20afec18e2e9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.890458 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.891358 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.891643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.891922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.892593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.894208 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.903954 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1277/2589] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:17.907688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.917535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f313be3-e769-403f-b9a4-879b756a3f87 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.922178 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.922800 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.923080 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.923353 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.923986 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.925560 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e87ae9-077d-4b5b-b043-174d471d9705 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:17.938301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1313/2590] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:17.940774 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:17.951571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ae2ef-2dc2-405a-aaa2-bb63800651b8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:17.956209 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:17.956882 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:17.957162 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:17.957437 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] PATH_INFO: `/v3/projects/2784551f5b0a4e73a15f0309db36e833/users/8938f498c66247c4966577c548b860b9/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:17.972957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2784551f5b0a4e73a15f0309db36e833, user_id=8938f498c66247c4966577c548b860b9, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:17.975616 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:18.071248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1278/2591] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:17 2022] PUT /identity/v3/projects/2784551f5b0a4e73a15f0309db36e833/users/8938f498c66247c4966577c548b860b9/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:18.082243 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:18.082569 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:18.082855 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:18.110872 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] MFA Rules not processed for user `8938f498c66247c4966577c548b860b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:18.112087 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:18.175421 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1314/2592] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:18.186414 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:18.188155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:18.244125 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd2eef-7952-4a92-bcaa-b15350ed963d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:18.248323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:18.248919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:18.249199 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:18.249469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:18.257552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:18.258995 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61daff-473c-4216-a34d-b306da600ab3 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:18.267763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1279/2593] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:28.172555 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:28.263763 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:29.227540 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:29.291262 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859c0df-81bc-49b8-9647-c450dbb683a3 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:29.294443 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:29.295092 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:29.295389 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:29.295663 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] PATH_INFO: `/v3/users/8938f498c66247c4966577c548b860b9` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:29.296762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8938f498c66247c4966577c548b860b9)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:29.298812 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:29.506166 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8938f498c66247c4966577c548b860b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:29.513239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8938f498c66247c4966577c548b860b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:29.525789 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8938f498c66247c4966577c548b860b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:29.526464 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8938f498c66247c4966577c548b860b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:29.575866 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8938f498c66247c4966577c548b860b9'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:29.582301 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1315/2594] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:29 2022] DELETE /identity/v3/users/8938f498c66247c4966577c548b860b9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:29.587043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:29.587553 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:29.587713 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:29.606802 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:29.608011 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:29.665058 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1280/2595] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:29.674807 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:29.676191 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:29.731463 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26c9d8-30a9-45d0-8fcd-9ba90e41b82f admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:29.735032 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:29.735626 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:29.736500 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:29.736778 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:29.745532 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:29.747004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff3c63-95d6-4d04-80c0-cb9391c84d4c service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:29.756706 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1316/2596] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:30.300714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:30.302242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:30.315226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fdd51-f190-4a05-b796-ba0e651695e3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:30.319795 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:30.320494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.320905 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.321034 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] PATH_INFO: `/v3/projects/2784551f5b0a4e73a15f0309db36e833` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.322094 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2784551f5b0a4e73a15f0309db36e833)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:30.333593 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:30.464741 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2784551f5b0a4e73a15f0309db36e833', 'request_id': 'req-1e1b58c5-7b31-413b-9c27-3429f87b284b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:30.470819 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2784551f5b0a4e73a15f0309db36e833', 'request_id': 'req-1e1b58c5-7b31-413b-9c27-3429f87b284b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:30.521717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2784551f5b0a4e73a15f0309db36e833', 'request_id': 'req-1e1b58c5-7b31-413b-9c27-3429f87b284b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:30.533342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] The token cache is being invalidate because project 2784551f5b0a4e73a15f0309db36e833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:30.533632 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:30.534259 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1e1b58c5-7b31-413b-9c27-3429f87b284b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:30.535743 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1281/2597] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:30 2022] DELETE /identity/v3/projects/2784551f5b0a4e73a15f0309db36e833 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:30.541060 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.541566 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.541687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.561229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:30.562416 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:30.578684 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:30.620133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1317/2598] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:30.628759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.630639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.630639 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.648723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:30.649915 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:30.658235 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1282/2599] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:30.662373 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:30.672420 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd080321-3526-4819-ba6e-b7512f46c3a5 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:30.677662 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:30.678260 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.678578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.678860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.679503 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 14:00:30.679830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:30.681482 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:30.693137 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1318/2600] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 14:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 14:00:30.697282 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:30.706386 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977398-1826-4b30-ada9-b4a62d5cfdf5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:30.711006 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:30.711592 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.711871 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.712146 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.712771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:30.714676 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:30.805846 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1283/2601] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:30 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 14:00:30.809969 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:30.819988 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71210e9b-0351-42f4-a5ea-c3cf19028783 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:30.875350 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:30.875943 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.876225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.876497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.877127 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:30.879058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:30.970945 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1319/2602] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:30 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:30.975139 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:30.985965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5095418-b9ab-48ab-bd4f-ac1bdda4cb64 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:30.990428 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:30.991052 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:30.991334 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:30.991609 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:30.992254 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:30.993877 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:31.003778 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1284/2603] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:31.007697 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:31.018053 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ba486-5c1e-4c2e-b35d-31f15b3d5523 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:31.022633 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:31.023229 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:31.023512 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:31.023784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:31.024430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:31.026004 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ec2a1-29ff-4e1e-afe5-93a9be59f20a admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:31.036957 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1320/2604] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:31.041210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:31.051585 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064abf-472d-481f-8f9d-7734ba5540d2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:31.056157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:31.056794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:31.057072 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:31.057342 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] PATH_INFO: `/v3/projects/1c479c77306c4eca9b17d544056a4500/users/8793c205b8b0432e947800b56cc447b2/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:31.071645 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c479c77306c4eca9b17d544056a4500, user_id=8793c205b8b0432e947800b56cc447b2, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:31.074232 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:31.147496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1285/2605] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:31 2022] PUT /identity/v3/projects/1c479c77306c4eca9b17d544056a4500/users/8793c205b8b0432e947800b56cc447b2/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:31.161240 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:31.161535 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:31.161818 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:31.189524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] MFA Rules not processed for user `8793c205b8b0432e947800b56cc447b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:31.190755 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:31.252678 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1321/2606] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:31.263392 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:31.265548 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:31.288771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:31.322594 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d0b9f-8c80-44f2-8fbb-0f2691a43053 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:31.326854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:31.327452 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:31.327730 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:31.328004 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:31.336808 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:31.338234 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb59deb-63bb-4adb-9515-744950584aa9 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:31.347389 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1286/2607] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:35.355769 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:35.374487 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:35.418062 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ec70-4034-405c-a21b-0e646d186773 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:35.422490 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:35.423179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:35.423542 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:35.423820 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] PATH_INFO: `/v3/users/8793c205b8b0432e947800b56cc447b2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:35.424960 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8793c205b8b0432e947800b56cc447b2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:35.426899 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:35.586452 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8793c205b8b0432e947800b56cc447b2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:35.594058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8793c205b8b0432e947800b56cc447b2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:35.605741 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8793c205b8b0432e947800b56cc447b2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:35.606411 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8793c205b8b0432e947800b56cc447b2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:35.675889 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8793c205b8b0432e947800b56cc447b2'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:35.681724 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1322/2608] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:35 2022] DELETE /identity/v3/users/8793c205b8b0432e947800b56cc447b2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:35.686188 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:35.686699 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:35.687336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:35.706236 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:35.707456 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:35.764669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1287/2609] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:35.775251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:35.776682 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:35.832788 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bfe7a-190d-4872-bee6-94548d709126 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:35.836430 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:35.837085 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:35.837315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:35.837589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:35.846451 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:35.847900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d09ce7-5d25-4f8f-b2ba-35889c92f71a service neutron] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:35.857157 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1323/2610] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:36.235195 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.236691 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:36.250480 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749e967-3a5a-4b5a-b177-4500ffc3327f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.256245 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.256946 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.257226 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.257496 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] PATH_INFO: `/v3/projects/1c479c77306c4eca9b17d544056a4500` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.258669 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c479c77306c4eca9b17d544056a4500)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.260552 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.362819 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c479c77306c4eca9b17d544056a4500', 'request_id': 'req-3ae1ce5e-e117-413a-b822-32fc42746c95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:36.368253 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c479c77306c4eca9b17d544056a4500', 'request_id': 'req-3ae1ce5e-e117-413a-b822-32fc42746c95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:36.427075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c479c77306c4eca9b17d544056a4500', 'request_id': 'req-3ae1ce5e-e117-413a-b822-32fc42746c95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:36.438780 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] The token cache is being invalidate because project 1c479c77306c4eca9b17d544056a4500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:36.439070 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:36.439719 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ae1ce5e-e117-413a-b822-32fc42746c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:36.441193 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1288/2611] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:36 2022] DELETE /identity/v3/projects/1c479c77306c4eca9b17d544056a4500 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:36.446225 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.446726 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.446831 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.467133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:36.468314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:36.502251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:36.527683 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1324/2612] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:36.536210 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.536495 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.536771 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.557700 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:36.563571 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:36.566767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1289/2613] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:36.570887 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.580362 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0836787-aed5-46cb-a2ac-fde8f3e5e90f None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.585414 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.586006 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.586287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.586595 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.587239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 14:00:36.587541 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.589115 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.600762 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1325/2614] 10.222.0.39 () {58 vars in 1153 bytes} [Wed Oct 19 14:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 14:00:36.604919 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.614684 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748308-df29-4b61-9d9c-39674ee22a61 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.619151 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.619734 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.620014 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.620287 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.620916 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.622839 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.696575 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1290/2615] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:36 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:36.700717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.710578 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98352a54-5ac8-4759-b563-bad0e80da1eb admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.769830 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.770544 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.770825 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.771103 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.771746 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.773725 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.905314 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1326/2616] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:36 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 14:00:36.909974 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.920748 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b6f2-f109-4161-ac34-231e8e8afd7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.925317 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.925964 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.926220 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.926494 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.927372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.928948 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.939608 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1291/2617] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:36.943745 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.954287 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5eab2-263b-4ee3-8821-557ceeeabe2c admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.958900 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.959496 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.959781 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.960052 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.960688 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:36.962259 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef96c7-d753-498a-ab45-47df5b83c7bc admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:36.973752 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1327/2618] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:36.977881 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:36.988242 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f5995-aa34-4416-886f-39da3aee0380 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:36.993155 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:36.993803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:36.994082 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:36.994354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] PATH_INFO: `/v3/projects/9b3d952f57364e718020d748f45dab1e/users/537c3278e9514a359f3e8ca2180c5ca7/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:36.996679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:37.014174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3d952f57364e718020d748f45dab1e, user_id=537c3278e9514a359f3e8ca2180c5ca7, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.014174 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.087515 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1292/2619] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:36 2022] PUT /identity/v3/projects/9b3d952f57364e718020d748f45dab1e/users/537c3278e9514a359f3e8ca2180c5ca7/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:37.099239 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.099524 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.099794 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.127661 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] MFA Rules not processed for user `537c3278e9514a359f3e8ca2180c5ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:37.128857 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:37.191846 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1328/2620] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:37.196216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.256776 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70557de-0d61-46c9-80fe-07843c7ca0ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.261349 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.261947 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.262229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.262522 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.263157 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.265036 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.376497 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1293/2621] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:37.381255 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.392141 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58f175-3ab0-4184-a24c-dc5e043281c8 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.423903 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:37.447427 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.448033 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.448224 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.448493 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.449187 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.451119 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.570361 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1329/2622] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:37.574345 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.584501 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcda75a-d0cd-4aca-acb3-9a9d6ae07270 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.589055 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.589647 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.589927 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.590216 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.590867 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.592469 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.603027 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1294/2623] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:37.606736 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.616860 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf9c54-d327-4894-aad0-9a2388e59ece admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.621581 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.622179 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.622461 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.622774 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] PATH_INFO: `/v3/roles` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.623415 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.625021 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbab5da-73b1-4a11-8902-bac181d4cdef admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.635588 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1330/2624] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:37.639388 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.649311 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d8e06-429b-40d8-9826-32cf152c37c3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.653816 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.654446 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.654768 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.655043 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] PATH_INFO: `/v3/projects/eca27d59353e40f6869699903d491949/users/1df99d799a5148129fd18f9ded07d1a1/roles/5eaea2af31a1491a9e7ad2736d630ee6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.669086 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca27d59353e40f6869699903d491949, user_id=1df99d799a5148129fd18f9ded07d1a1, role_id=5eaea2af31a1491a9e7ad2736d630ee6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.671703 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.724012 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1295/2625] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:37 2022] PUT /identity/v3/projects/eca27d59353e40f6869699903d491949/users/1df99d799a5148129fd18f9ded07d1a1/roles/5eaea2af31a1491a9e7ad2736d630ee6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:37.734022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.734316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.734711 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.763047 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] MFA Rules not processed for user `1df99d799a5148129fd18f9ded07d1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:37.764261 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:37.826759 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1331/2626] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:37.831249 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.892445 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8790a-998f-413c-aafd-6128f52954c6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:37.897111 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:37.897710 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:37.897993 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:37.898271 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:37.898918 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:37.900794 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:37.980368 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1296/2627] 10.222.0.39 () {60 vars in 1153 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:37.984502 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:37.995776 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ff3a9-7825-48c6-9f6d-7290d482e485 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.053152 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.053760 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.054042 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.054315 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.054982 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.056884 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.135144 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1332/2628] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Oct 19 14:00:37 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 14:00:38.139223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:38.149792 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ac53-439c-42ef-a7b2-304cec983632 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.154491 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.159292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.159292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.159292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.159292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.159292 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.169075 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1297/2629] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:38.173133 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:38.183442 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27961c-0e7f-4e6c-8dc2-d74b9df9b9d9 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.188016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.188660 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.188963 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.189248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] PATH_INFO: `/v3/projects/b3b226bc589b4e33a30f00df2290e8fe/users/866c31c1eb73461a94e71388b3c18eff/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.203644 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b226bc589b4e33a30f00df2290e8fe, user_id=866c31c1eb73461a94e71388b3c18eff, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.206316 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.275784 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1333/2630] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:38 2022] PUT /identity/v3/projects/b3b226bc589b4e33a30f00df2290e8fe/users/866c31c1eb73461a94e71388b3c18eff/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:38.279767 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:38.290914 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9c17-b95a-44c7-bc05-08bb338a2176 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.294420 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:38.347724 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.348323 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.348606 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.348932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.349598 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.351296 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089d892-6ffd-4364-bf19-ece393d0e10a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.360981 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1298/2631] 10.222.0.39 () {58 vars in 1122 bytes} [Wed Oct 19 14:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 14:00:38.365122 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:38.375400 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf207d3-ff79-4010-984b-ac5f979c0344 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.380058 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.380692 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.380974 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.381247 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] PATH_INFO: `/v3/projects/b3b226bc589b4e33a30f00df2290e8fe/users/866c31c1eb73461a94e71388b3c18eff/roles/821e0e3786404ca6a02982ae9bd496a2` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.383230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b226bc589b4e33a30f00df2290e8fe, user_id=866c31c1eb73461a94e71388b3c18eff, role_id=821e0e3786404ca6a02982ae9bd496a2)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.385833 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.388842 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:38.396989 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1334/2632] 10.222.0.39 () {60 vars in 1483 bytes} [Wed Oct 19 14:00:38 2022] PUT /identity/v3/projects/b3b226bc589b4e33a30f00df2290e8fe/users/866c31c1eb73461a94e71388b3c18eff/roles/821e0e3786404ca6a02982ae9bd496a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:38.406890 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.407179 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.407472 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.435717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] MFA Rules not processed for user `866c31c1eb73461a94e71388b3c18eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:38.436922 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:38.502649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1299/2633] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:38.513811 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:38.515862 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:38.573277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f40b3-75fb-4fc6-8f02-451053eb6e96 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:38.576880 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:38.577472 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:38.577748 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:38.578016 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:38.631839 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:38.633251 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:38.642027 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1335/2634] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:40.585277 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:40.586759 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:40.598854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809504-cf1d-4560-b157-05df93978a54 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:40.602346 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:40.607803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:40.607803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:40.607803 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:40.660587 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:40.662001 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:40.670558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1300/2635] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:42.125470 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:42.133987 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7a4d7-6f2d-4708-b45b-2dd624ca8644 service cinder] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:42.137520 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf272634ade542739c5cfe3139bb201a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:42.138114 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] REQUEST_METHOD: `GET` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:42.138395 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:42.138704 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:42.148008 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:42.149425 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143d83-6ed6-4146-8ac9-a7da201a2893 service cinder] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:42.159231 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1336/2636] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:46.799643 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:46.829541 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:46.862171 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fec8-b27a-464d-9af1-8902206ce488 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:46.867029 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:46.867650 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:46.867932 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:46.868204 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] PATH_INFO: `/v3/users/537c3278e9514a359f3e8ca2180c5ca7` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:46.869305 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537c3278e9514a359f3e8ca2180c5ca7)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:46.871247 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:47.032229 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537c3278e9514a359f3e8ca2180c5ca7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.033372 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537c3278e9514a359f3e8ca2180c5ca7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.041336 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537c3278e9514a359f3e8ca2180c5ca7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.053144 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537c3278e9514a359f3e8ca2180c5ca7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.109773 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537c3278e9514a359f3e8ca2180c5ca7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.116395 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1301/2637] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:46 2022] DELETE /identity/v3/users/537c3278e9514a359f3e8ca2180c5ca7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:47.120979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] REQUEST_METHOD: `POST` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:47.121465 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:47.121572 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:47.140565 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.auth.core [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] MFA Rules not processed for user `94146067898240bfb358cef661f9b970`. Rule list: `[]` (Enabled: `True`). {{(pid=81538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 14:00:47.141798 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:47.198351 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1337/2638] 10.222.0.39 () {58 vars in 958 bytes} [Wed Oct 19 14:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 14:00:47.207960 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:47.209348 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:47.265717 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02110-fc81-4ff5-88e4-ebdad874d440 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:47.269758 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f2a170d431b4524b18178757fa3d627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:47.270354 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:47.270664 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:47.270942 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:47.279723 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:47.281130 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:47.290944 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1302/2639] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:00:47.694234 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:47.695728 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:47.708124 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c12e5-42c8-4885-b56d-a10115dfaea2 None None] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:47.712771 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:47.713409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:47.713693 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:47.713970 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] PATH_INFO: `/v3/projects/9b3d952f57364e718020d748f45dab1e` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:47.715107 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3d952f57364e718020d748f45dab1e)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:47.717022 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:47.818589 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:47.830750 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3d952f57364e718020d748f45dab1e', 'request_id': 'req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.836347 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3d952f57364e718020d748f45dab1e', 'request_id': 'req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.915522 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3d952f57364e718020d748f45dab1e', 'request_id': 'req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.927681 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] The token cache is being invalidate because project 9b3d952f57364e718020d748f45dab1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:47.928161 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.929109 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:47.930630 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1338/2640] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:47 2022] DELETE /identity/v3/projects/9b3d952f57364e718020d748f45dab1e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:47.934619 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:47.944985 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02945b7-f781-42b5-9413-7d0292bae40d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:47.989611 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:48.001973 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:48.002658 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:48.002945 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:48.003221 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] PATH_INFO: `/v3/users/1df99d799a5148129fd18f9ded07d1a1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:48.004355 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df99d799a5148129fd18f9ded07d1a1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:48.006329 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:48.204136 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df99d799a5148129fd18f9ded07d1a1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.204704 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df99d799a5148129fd18f9ded07d1a1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.211769 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df99d799a5148129fd18f9ded07d1a1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.223649 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df99d799a5148129fd18f9ded07d1a1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.292854 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df99d799a5148129fd18f9ded07d1a1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.299324 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1303/2641] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:47 2022] DELETE /identity/v3/users/1df99d799a5148129fd18f9ded07d1a1 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:48.682687 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:48.684230 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:48.698668 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa173aa-e56b-4c97-9c06-155e2eda2aa0 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:48.755679 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:48.756248 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:48.756533 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:48.756814 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] PATH_INFO: `/v3/projects/eca27d59353e40f6869699903d491949` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:48.757932 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca27d59353e40f6869699903d491949)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:48.759883 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:48.844029 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca27d59353e40f6869699903d491949', 'request_id': 'req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.850328 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca27d59353e40f6869699903d491949', 'request_id': 'req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.918194 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca27d59353e40f6869699903d491949', 'request_id': 'req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.929700 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] The token cache is being invalidate because project eca27d59353e40f6869699903d491949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:48.929992 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.930680 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:48.932154 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1339/2642] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:48 2022] DELETE /identity/v3/projects/eca27d59353e40f6869699903d491949 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:48.936101 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:48.937514 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:48.949910 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72fb11-4e30-4a91-898b-25fa4b33e024 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:49.005301 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:49.005989 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:49.006215 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:49.006485 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] PATH_INFO: `/v3/users/866c31c1eb73461a94e71388b3c18eff` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:49.007659 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=866c31c1eb73461a94e71388b3c18eff)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:49.009558 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:49.160135 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866c31c1eb73461a94e71388b3c18eff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.160788 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866c31c1eb73461a94e71388b3c18eff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.167800 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866c31c1eb73461a94e71388b3c18eff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.177934 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:49.180203 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866c31c1eb73461a94e71388b3c18eff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.241248 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866c31c1eb73461a94e71388b3c18eff'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.247223 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1304/2643] 10.222.0.39 () {58 vars in 1224 bytes} [Wed Oct 19 14:00:48 2022] DELETE /identity/v3/users/866c31c1eb73461a94e71388b3c18eff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:49.631737 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Authenticating user token {{(pid=81538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:00:49.633188 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.fernet_utils [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:00:49.646277 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf92bb-24d2-43d9-9f96-c5357d0bcf79 admin admin] Validating token access rules against request {{(pid=81538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:00:49.693717 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:49.703342 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94146067898240bfb358cef661f9b970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf3ff3a529704cf9a1e0bae3d4c303ad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:00:49.703979 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:00:49.704374 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:00:49.704634 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] PATH_INFO: `/v3/projects/b3b226bc589b4e33a30f00df2290e8fe` {{(pid=81538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:00:49.705763 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b226bc589b4e33a30f00df2290e8fe)` {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:00:49.707727 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] RBAC: Authorization granted {{(pid=81538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:00:49.828160 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b226bc589b4e33a30f00df2290e8fe', 'request_id': 'req-d3661b4c-17bb-402a-9a7d-2eb02067a680'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.833638 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b226bc589b4e33a30f00df2290e8fe', 'request_id': 'req-d3661b4c-17bb-402a-9a7d-2eb02067a680'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.895497 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b226bc589b4e33a30f00df2290e8fe', 'request_id': 'req-d3661b4c-17bb-402a-9a7d-2eb02067a680'} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.906991 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] The token cache is being invalidate because project b3b226bc589b4e33a30f00df2290e8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 14:00:49.907773 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.908409 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG keystone.notifications [None req-d3661b4c-17bb-402a-9a7d-2eb02067a680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 14:00:49.909895 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: [pid: 81538|app: 0|req: 1340/2644] 10.222.0.39 () {58 vars in 1233 bytes} [Wed Oct 19 14:00:49 2022] DELETE /identity/v3/projects/b3b226bc589b4e33a30f00df2290e8fe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 14:00:59.292965 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=2,keystone:SELECT=3,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:00:59.903938 dv-cinder-861883-1-iscsi devstack@keystone.service[81538]: DEBUG dbcounter [-] [81538] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 14:04:20.428122 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 14:04:20.433695 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:04:20.567083 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82a0e-6f6c-4adc-9a2b-a7f3be59e83f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 14:04:20.624089 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99786bcc97fa4b7eb9141fb3d509c28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60cab2a8445f4898bae2696caebbb468', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 14:04:20.624679 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 14:04:20.624957 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 14:04:20.625230 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 14:04:20.626968 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 14:04:20.638128 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 14:04:20.639714 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82273eb5-08fe-4458-948b-a584d7f19b36 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 14:04:20.650108 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1305/2645] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Oct 19 14:04:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 14:04:30.645850 dv-cinder-861883-1-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}